[Zope] Important Fix for Zope 2.0 through 2.1.6
Evan Simpson
evan@digicool.com
Sun, 10 Dec 2000 11:13:36 -0500
This is a multi-part message in MIME format.
------=_NextPart_000_0045_01C0629A.3D5D3C60
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Thanks to Jeff Ragsdale, we've finally been able to kill a
longstanding bug that allows POST requests to interfere with
each other. Symptoms include corrupted or aborted File and
Image uploads, and stupid-log messages about
"AttributeError: data" killing threads.
The attached HTTPServer.py is valid for all Zope 2.0.x and
2.1.x versions. I am posting the patched file for Zope
2.2.x separately.
PLEASE BACK UP <Zope>/ZServer/HTTPServer.py, then replace it
with the attached file.
Cheers,
Evan @ digicool & 4-am
------=_NextPart_000_0045_01C0629A.3D5D3C60
Content-Type: text/plain;
name="HTTPServer.py"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="HTTPServer.py"
#########################################################################=
#####=0A=
# =0A=
# Zope Public License (ZPL) Version 1.0=0A=
# -------------------------------------=0A=
# =0A=
# Copyright (c) Digital Creations. All rights reserved.=0A=
# =0A=
# This license has been certified as Open Source(tm).=0A=
# =0A=
# Redistribution and use in source and binary forms, with or without=0A=
# modification, are permitted provided that the following conditions are=0A=
# met:=0A=
# =0A=
# 1. Redistributions in source code must retain the above copyright=0A=
# notice, this list of conditions, and the following disclaimer.=0A=
# =0A=
# 2. Redistributions in binary form must reproduce the above copyright=0A=
# notice, this list of conditions, and the following disclaimer in=0A=
# the documentation and/or other materials provided with the=0A=
# distribution.=0A=
# =0A=
# 3. Digital Creations requests that attribution be given to Zope=0A=
# in any manner possible. Zope includes a "Powered by Zope"=0A=
# button that is installed by default. While it is not a license=0A=
# violation to remove this button, it is requested that the=0A=
# attribution remain. A significant investment has been put=0A=
# into Zope, and this effort will continue if the Zope community=0A=
# continues to grow. This is one way to assure that growth.=0A=
# =0A=
# 4. All advertising materials and documentation mentioning=0A=
# features derived from or use of this software must display=0A=
# the following acknowledgement:=0A=
# =0A=
# "This product includes software developed by Digital Creations=0A=
# for use in the Z Object Publishing Environment=0A=
# (http://www.zope.org/)."=0A=
# =0A=
# In the event that the product being advertised includes an=0A=
# intact Zope distribution (with copyright and license included)=0A=
# then this clause is waived.=0A=
# =0A=
# 5. Names associated with Zope or Digital Creations must not be used to=0A=
# endorse or promote products derived from this software without=0A=
# prior written permission from Digital Creations.=0A=
# =0A=
# 6. Modified redistributions of any form whatsoever must retain=0A=
# the following acknowledgment:=0A=
# =0A=
# "This product includes software developed by Digital Creations=0A=
# for use in the Z Object Publishing Environment=0A=
# (http://www.zope.org/)."=0A=
# =0A=
# Intact (re-)distributions of any official Zope release do not=0A=
# require an external acknowledgement.=0A=
# =0A=
# 7. Modifications are encouraged but must be packaged separately as=0A=
# patches to official Zope releases. Distributions that do not=0A=
# clearly separate the patches from the original work must be clearly=0A=
# labeled as unofficial distributions. Modifications which do not=0A=
# carry the name Zope may be packaged in any form, as long as they=0A=
# conform to all of the clauses above.=0A=
# =0A=
# =0A=
# Disclaimer=0A=
# =0A=
# THIS SOFTWARE IS PROVIDED BY DIGITAL CREATIONS ``AS IS'' AND ANY=0A=
# EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE=0A=
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR=0A=
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL DIGITAL CREATIONS OR ITS=0A=
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,=0A=
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT=0A=
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF=0A=
# USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND=0A=
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,=0A=
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT=0A=
# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF=0A=
# SUCH DAMAGE.=0A=
# =0A=
# =0A=
# This software consists of contributions made by Digital Creations and=0A=
# many individuals on behalf of Digital Creations. Specific=0A=
# attributions are listed in the accompanying credits file.=0A=
# =0A=
#########################################################################=
#####=0A=
=0A=
"""=0A=
Medusa HTTP server for Zope=0A=
=0A=
changes from Medusa's http_server=0A=
=0A=
Request Threads -- Requests are processed by threads from a thread=0A=
pool.=0A=
=0A=
Output Handling -- Output is pushed directly into the producer=0A=
fifo by the request-handling thread. The HTTP server does not do=0A=
any post-processing such as chunking.=0A=
=0A=
Pipelineable -- This is needed for protocols such as HTTP/1.1 in=0A=
which mutiple requests come in on the same channel, before=0A=
responses are sent back. When requests are pipelined, the client=0A=
doesn't wait for the response before sending another request. The=0A=
server must ensure that responses are sent back in the same order=0A=
as requests are received.=0A=
=0A=
""" =0A=
import sys=0A=
import regex=0A=
import string=0A=
import os=0A=
import types=0A=
import thread=0A=
import time=0A=
from cStringIO import StringIO=0A=
=0A=
from PubCore import handle=0A=
from HTTPResponse import make_response=0A=
from ZPublisher.HTTPRequest import HTTPRequest=0A=
=0A=
from medusa.http_server import http_server, http_channel=0A=
from medusa import counter, producers, asyncore, max_sockets=0A=
from medusa.default_handler import split_path, unquote, get_header=0A=
from medusa.asyncore import compact_traceback, dispatcher=0A=
=0A=
from ZServer import CONNECTION_LIMIT, ZOPE_VERSION, ZSERVER_VERSION=0A=
=0A=
from zLOG import LOG, register_subsystem, BLATHER, INFO, WARNING, ERROR=0A=
import DebugLogger=0A=
=0A=
register_subsystem('ZServer HTTPServer')=0A=
=0A=
CONTENT_LENGTH =3D regex.compile('Content-Length: =
\([0-9]+\)',regex.casefold)=0A=
CONNECTION =3D regex.compile ('Connection: \(.*\)', regex.casefold)=0A=
=0A=
# maps request some headers to environment variables.=0A=
# (those that don't start with 'HTTP_')=0A=
header2env=3D{'content-length' : 'CONTENT_LENGTH',=0A=
'content-type' : 'CONTENT_TYPE',=0A=
'connection' : 'CONNECTION_TYPE',=0A=
}=0A=
=0A=
class zhttp_collector:=0A=
def __init__(self, handler, request, size):=0A=
self.handler =3D handler=0A=
self.request =3D request=0A=
if size > 524288:=0A=
# write large upload data to a file=0A=
from tempfile import TemporaryFile=0A=
self.data =3D TemporaryFile('w+b')=0A=
else:=0A=
self.data =3D StringIO()=0A=
request.channel.set_terminator(size)=0A=
request.collector=3Dself=0A=
=0A=
# put and post collection methods=0A=
#=0A=
def collect_incoming_data (self, data):=0A=
self.data.write(data)=0A=
=0A=
def found_terminator(self):=0A=
# reset collector=0A=
self.request.channel.set_terminator('\r\n\r\n')=0A=
self.request.collector=3DNone=0A=
# finish request=0A=
self.data.seek(0)=0A=
r=3Dself.request=0A=
d=3Dself.data=0A=
del self.request=0A=
del self.data=0A=
self.handler.continue_request(d,r)=0A=
=0A=
class zhttp_handler:=0A=
"A medusa style handler for zhttp_server"=0A=
=0A=
def __init__ (self, module, uri_base=3DNone, env=3DNone):=0A=
"""Creates a zope_handler=0A=
=0A=
module -- string, the name of the module to publish=0A=
uri_base -- string, the base uri of the published module=0A=
defaults to '/<module name>' if not given.=0A=
env -- dictionary, environment variables to be overridden. =
=0A=
Replaces standard variables with supplied ones.=0A=
"""=0A=
=0A=
self.module_name=3Dmodule=0A=
self.env_override=3Denv or {}=0A=
self.hits =3D counter.counter()=0A=
# if uri_base is unspecified, assume it=0A=
# starts with the published module name=0A=
#=0A=
if uri_base is None:=0A=
uri_base=3D'/%s' % module=0A=
elif uri_base =3D=3D '':=0A=
uri_base=3D'/'=0A=
else:=0A=
if uri_base[0] !=3D '/':=0A=
uri_base=3D'/'+uri_base=0A=
if uri_base[-1] =3D=3D '/':=0A=
uri_base=3Duri_base[:-1]=0A=
self.uri_base=3Duri_base=0A=
uri_regex=3D'%s.*' % self.uri_base=0A=
self.uri_regex =3D regex.compile(uri_regex)=0A=
=0A=
def match(self, request):=0A=
uri =3D request.uri=0A=
if self.uri_regex.match(uri) =3D=3D len(uri):=0A=
return 1=0A=
else:=0A=
return 0=0A=
=0A=
def handle_request(self,request):=0A=
self.hits.increment()=0A=
=0A=
DebugLogger.log('B', id(request), '%s %s' % =
(string.upper(request.command), request.uri))=0A=
=0A=
size=3Dget_header(CONTENT_LENGTH, request.header)=0A=
if size and size !=3D '0':=0A=
size=3Dstring.atoi(size)=0A=
zhttp_collector(self, request, size)=0A=
else:=0A=
sin=3DStringIO()=0A=
self.continue_request(sin,request)=0A=
=0A=
def get_environment(self, request,=0A=
# These are strictly performance hackery...=0A=
split=3Dstring.split,=0A=
strip=3Dstring.strip,=0A=
join =3Dstring.join,=0A=
upper=3Dstring.upper,=0A=
lower=3Dstring.lower,=0A=
h2ehas=3Dheader2env.has_key,=0A=
h2eget=3Dheader2env.get,=0A=
workdir=3Dos.getcwd(),=0A=
ospath=3Dos.path,=0A=
):=0A=
[path, params, query, fragment] =3D split_path(request.uri)=0A=
while path and path[0] =3D=3D '/':=0A=
path =3D path[1:]=0A=
if '%' in path:=0A=
path =3D unquote(path)=0A=
if query:=0A=
# ZPublisher doesn't want the leading '?'=0A=
query =3D query[1:]=0A=
=0A=
server=3Drequest.channel.server=0A=
env =3D {}=0A=
env['REQUEST_METHOD']=3Dupper(request.command)=0A=
env['SERVER_PORT']=3Dstr(server.port)=0A=
env['SERVER_NAME']=3Dserver.server_name=0A=
env['SERVER_SOFTWARE']=3Dserver.SERVER_IDENT=0A=
env['SERVER_PROTOCOL']=3Drequest.version=0A=
env['channel.creation_time']=3Drequest.channel.creation_time=0A=
if self.uri_base=3D=3D'/':=0A=
env['SCRIPT_NAME']=3D''=0A=
env['PATH_INFO']=3D'/' + path=0A=
else:=0A=
env['SCRIPT_NAME'] =3D self.uri_base=0A=
try:=0A=
path_info=3Dsplit(path,self.uri_base[1:],1)[1]=0A=
except:=0A=
path_info=3D''=0A=
env['PATH_INFO']=3Dpath_info=0A=
env['PATH_TRANSLATED']=3Dospath.normpath(ospath.join(=0A=
workdir, env['PATH_INFO']))=0A=
if query:=0A=
env['QUERY_STRING'] =3D query=0A=
env['GATEWAY_INTERFACE']=3D'CGI/1.1'=0A=
env['REMOTE_ADDR']=3Drequest.channel.addr[0]=0A=
=0A=
# If we're using a resolving logger, try to get the=0A=
# remote host from the resolver's cache.=0A=
if hasattr(server.logger, 'resolver'):=0A=
dns_cache=3Dserver.logger.resolver.cache=0A=
if dns_cache.has_key(env['REMOTE_ADDR']):=0A=
remote_host=3Ddns_cache[env['REMOTE_ADDR']][2]=0A=
if remote_host is not None:=0A=
env['REMOTE_HOST']=3Dremote_host=0A=
=0A=
env_has=3Denv.has_key=0A=
for header in request.header:=0A=
key,value=3Dsplit(header,":",1)=0A=
key=3Dlower(key)=0A=
value=3Dstrip(value)=0A=
if h2ehas(key) and value:=0A=
env[h2eget(key)]=3Dvalue=0A=
else:=0A=
key=3D'HTTP_%s' % upper(join(split(key, "-"), "_"))=0A=
if value and not env_has(key):=0A=
env[key]=3Dvalue=0A=
env.update(self.env_override)=0A=
return env=0A=
=0A=
def continue_request(self, sin, request):=0A=
"continue handling request now that we have the stdin"=0A=
=0A=
s=3Dget_header(CONTENT_LENGTH, request.header)=0A=
if s:=0A=
s=3Dstring.atoi(s)=0A=
else:=0A=
s=3D0 =0A=
DebugLogger.log('I', id(request), s)=0A=
=0A=
env=3Dself.get_environment(request)=0A=
zresponse=3Dmake_response(request,env)=0A=
zrequest=3DHTTPRequest(sin, env, zresponse)=0A=
request.channel.current_request=3DNone=0A=
request.channel.queue.append(self.module_name, zrequest, =
zresponse)=0A=
request.channel.work()=0A=
=0A=
def status(self):=0A=
return producers.simple_producer("""=0A=
<li>Zope Handler=0A=
<ul>=0A=
<li><b>Published Module:</b> % s=0A=
<li><b>Hits:</b> %d=0A=
</ul>""" %(self.module_name,int(self.hits))=0A=
)=0A=
=0A=
=0A=
=0A=
class zhttp_channel(http_channel):=0A=
"http channel"=0A=
=0A=
closed=3D0=0A=
zombie_timeout=3D100*60 # 100 minutes=0A=
=0A=
def __init__(self, server, conn, addr):=0A=
http_channel.__init__(self, server, conn, addr)=0A=
self.queue=3D[]=0A=
self.working=3D0=0A=
=0A=
def push(self, producer, send=3D1):=0A=
# this is thread-safe when send is false=0A=
# note, that strings are not wrapped in =0A=
# producers by default=0A=
if self.closed:=0A=
return=0A=
self.producer_fifo.push(producer)=0A=
if send: self.initiate_send()=0A=
=0A=
push_with_producer=3Dpush=0A=
=0A=
def work(self):=0A=
"try to handle a request"=0A=
if not self.working:=0A=
if self.queue:=0A=
self.working=3D1=0A=
try: module_name, request, response=3Dself.queue.pop(0)=0A=
except: return=0A=
handle(module_name, request, response)=0A=
=0A=
def close(self):=0A=
self.closed=3D1=0A=
while self.queue:=0A=
self.queue.pop()=0A=
if self.current_request is not None:=0A=
self.current_request.channel=3DNone # break circ refs=0A=
self.current_request=3DNone=0A=
while self.producer_fifo:=0A=
p=3Dself.producer_fifo.first()=0A=
if p is not None and type(p) !=3D types.StringType:=0A=
p.more() # free up resources held by producer=0A=
self.producer_fifo.pop()=0A=
dispatcher.close(self)=0A=
=0A=
def done(self):=0A=
"Called when a publishing request is finished"=0A=
self.working=3D0=0A=
self.work()=0A=
=0A=
def kill_zombies(self):=0A=
now =3D int (time.time())=0A=
for channel in asyncore.socket_map.keys():=0A=
if channel.__class__ =3D=3D self.__class__:=0A=
if (now - channel.creation_time) > =
channel.zombie_timeout:=0A=
channel.close()=0A=
=0A=
=0A=
class zhttp_server(http_server): =0A=
"http server"=0A=
=0A=
SERVER_IDENT=3D'Zope/%s ZServer/%s' % (ZOPE_VERSION,ZSERVER_VERSION)=0A=
=0A=
channel_class =3D zhttp_channel=0A=
=0A=
def readable(self):=0A=
return self.accepting and \=0A=
len(asyncore.socket_map) < CONNECTION_LIMIT=0A=
=0A=
def listen(self, num):=0A=
# override asyncore limits for nt's listen queue size=0A=
self.accepting =3D 1=0A=
return self.socket.listen (num)=0A=
=0A=
------=_NextPart_000_0045_01C0629A.3D5D3C60--