Fw: [Zope] name based virtual hosts with squid

sebas ICK sebas@ick-kupplungen.de
Thu, 15 Nov 2001 02:21:51 -0800


For the mailing list archives ...


sebas.
----- Original Message -----
From: "Richard Barrett" <R.Barrett@ftel.co.uk>
To: "sebas ICK" <sebas@ick-kupplungen.de>
Sent: Wednesday, November 14, 2001 11:59 AM
Subject: Re: [Zope] name based virtual hosts with squid


At 16:00 14/11/2001 -0800, sebas ICK wrote:
>I want to use squid as a front end server indeed and hide zope and apache
>behind it. Squid can do some security checks and advanced caching for
>frequently requested site.
>
>I wonder why it is this badly (maybe not at all) documented how to handle
>multiple backend servers with squid.

Well, I guess  that's the trouble with open software.

>This a cleaned up version of the *relevant* part of my squid.conf:
>
>-------------------------------
>
># set port where squid listens to, enable logging
>http_port 131.174.168.120:80
>emulate_httpd_log on
>
># set security checks off for debugging purposes
>http_access allow all
>
># define virtual hosts for zope and apache
>httpd_accel_uses_host_header on
>httpd_accel_host zopedev.dhs.org   #redirect this to apache on port 90
>httpd_accel_port 90
>httpd_accel_host zopedev-alpha.dhs.org   #redirect this to zope on port
8080
>httpd_accel_port 8080
>

I'll say from the start that my experience is with using Apache to serve
its own content and transparently proxy for Zope servers. My interest in
Squid has been using it as a regular (non-transparent) outbound proxy
server for browsers running on machines inside our firewall. Still her goes.

Looking at the Squid documentation
(http://squid.visolve.com/squid24s1/contents.htm) I think I see a couple of
problems with your squid.conf:

1. under httpd_accel_host it says:

"This tag is used to set the hostname of the accelerated server. It's
possible to have only one destination server, so you can have only one
occurrence of this line. If you are going accelerate more than one server,
or transparent traffic, you will have to use the word virtual instead of a
hostname here"

   Note: "only one occurence", you are trying to specify two

2. under httpd_accel_port it says:

"Accelerated requests can only be forwarded to one port: there is no table
that associates accelerated hosts and a destination port. Squid will
connect to the port that you set the httpd_accel_port value to ... If you
want virtual port support then specify the port as "0"

    Note: "one port", you are trying to specify two

Can we be clear about what you are trying to achieve. I'll make the
following assumptions. If they are wrong then the argument collapses and
will have to be remade on whatever set of objectives you do wish to pursue:

1. You want to run Squid on a machine with the IP number 131.174.168.120,
listening on port 80.

2. You want Squid to act as a transparent proxy for Zope and Apache
servers. From what you've said, it isn't clear whether these other servers
are to be running on the same or different machines to Squid.

3. Each HTTP Request directed at 131.174.168.120:80 is to be passed
transparently to one of the "hidden" Zope/Apache servers with the server
chosen by analysis of the request headers.

4. Whether the hidden servers are running on the same machine as Squid or
not will affect how incoming URLs must be structured. A separate Redirector
program that you must identify to Squid has to take the incoming URL and,
based on its contents, rewrite it to a URL suitable for one of the hidden
servers. If the other servers are on the same machine but differentiable by
the Host header of the request, the Redirector can potentially use that to
rewrite the URL to reflect the different port number used by each server.
If however, the servers are on different machines and cannot be
differentiated by Host header then some part of the path in the URL will
need to allow the identification of which server is to get the rewritten
request.

By my reckoning your squid.conf has to contain something like this:

# set port where squid listens to, enable logging
http_port 131.174.168.120:80
emulate_httpd_log on

# set security checks off for debugging purposes
http_access allow all

# we are using virtual hosts
httpd_accel_uses_host_header on
httpd_accel_host virtual
httpd_accel_port 0

# and defining the redirector which will rewrite the incoming URLs
redirector_access allow
redirect_program /path/to/you/redirector/program

You need to consider the type of HTTP clients that will be in use. The Host
header is an HTTP/1.1 feature which some older clients may not provide and
a dependency on it in your redirector might bring you grief. A safer bet
might be to use the path in the incoming URL as the means of selecting the
server as this is HTTP version independent.

Another point to consider is the issue of dealing with re-writing redirects
from the hidden servers, achieving the equivalent of Apache's
ProxyPassReverse directive with Squid. Possibly a bigger problem is
checking for and rewriting returned content from the real servers which
contains non-relative URLs that do not take account of the proxying that is
taking place. For instance, how, with Apache, do you intend to resolve the
sort of issues that are overcome by the use of Zope's Virtual Host Monster.


>-------------------------------
>
>Maybe someone got another approach?
>
>kind regards,
>
>sebas.
>

If you are looking for simple solutions I'd urge you to consider using
Apache as the primary server for its own content and which can also
transparently proxy for Zope using ProxyPass, ProxyPassReverse and
RewriteCond/RewriteRule in the Apache httpd.conf. Zope's VHM solves much of
the content rewriting issue at source.

Best of luck

Richard

>----- Original Message -----
>Sent: Wednesday, November 14, 2001 3:10 AM
>Subject: Re: [Zope] name based virtual hosts with squid
>
>
>If I understand you correctly, you want to hide Apache and Zope behind
>Squid. If you want Squid to rewrite incoming URLs as appropriate to either
>the Apache or Zope server you will need to provide what is referred to in
>Squid documentation  as a Redirector program. This program/script, which
>you must supply, can rewrite the incoming URLs using whatever
>transformation rules you choose to program. The effect is equivalent to
>using ProxyPass and RewriteCond/RewriteRule with Apache.
>
>See the Squid documentation http://www.squid-cache.org/Doc/FAQ/FAQ-15.html
>for more information.
>
>Then again why use Squid? You could achieve similar objectives by having
>Apache ProxyPass/Rewrite directives to rewrite URLs to the Zope server. See
>this list's archives for postings on this subject. Then again Squid would
>probably work better with a high traffic levels.
>
>At 22:13 13/11/2001 -0800, Sebastian Kügler wrote:
> >Hi group,
> >
> >Here's my problem, maybe someone's got better idea's ...
> >
> >I want to use squid to server apache and zope on one port. domains under
> >zope.domain.org have tp be served by zope and domains with www.domain.org
>by
> >apache. I have the following in my squid.conf;
> >
> >-----------------------------------------------------
> >
> >httpd_accel_uses_host_header on
> ># added port 80 here to listen to port 80 (doH!) edited by sebas
> >  http_port 80
> >  emulate_httpd_log on     # use NCSA https style logfile
> >
> ># http_access allow all
> >  httpd_accel_host zopedev.dhs.org
> >  httpd_accel_host virtual
> >  httpd_accel_port 90    # port where Apache is running
> >
> ># added port 80 here to listen to port 80 (doH!) edited by sebas
> >  http_port 80
> >  emulate_httpd_log on     # use NCSA https style logfile
> ># this line replaces 'http_access deny all'
> ># http_access allow all
> >  httpd_accel_host zopedev-alpha.dhs.org
> >  httpd_accel_host virtual
> >  httpd_accel_port 8080    # port where Zope is running
> >
> >-------------------------------------------------------
> >
> >As some of you might already have guessed: it does not work in the right
> >way, squid does only seem to use the last http_port specified, just
forgets
> >about our first one.
> >
> >I tried different things with that config file, but none of them did the
> >trick.
> >
> >Maybe some of you got it working, if so please tell me how.
> >
> >regards,
> >
> >sebas.
> >
> >
> >
> >_______________________________________________
> >Zope maillist  -  Zope@zope.org
> >http://lists.zope.org/mailman/listinfo/zope
> >**   No cross posts or HTML encoding!  **
> >(Related lists -
> >  http://lists.zope.org/mailman/listinfo/zope-announce
> >  http://lists.zope.org/mailman/listinfo/zope-dev )