[Zope-dev] funky side-effects, possible bug in HTTPRequest.py

Jamie Heilman jamie@audible.transient.net
Fri, 20 Jun 2003 03:07:18 -0700


The last few days I've been working on patch to Zope 2 that will clean
up the textarea size preference handling in the ZMI.  Right now, its a
mess.  I kept running into this really irritating behavior such that,
when ever I'd go to pull something out of a request object, it'd end
up being found in the 'other' dictionary instead of where I expected
to be (which happened to be 'form').  After getting really curious as
to why this was happening I've managed to track it down to subtle a
change in the way that the HTTPRequest.get method worked between
revisions 1.75 and 1.74 (1.75 being mj's merge of the value tainting
code).  The code responsible (assuming the value isn't tainted):

        # Untrusted data *after* trusted data
        v = self.form.get(key, _marker)
        if v is not _marker:
            other[key] = v                      # *boom*
            return v

That magical promotion of the key & value to the other dictionary is
what tripped me up.  This technique, while originally used only for
known convenience variables like URLx or BASEx and for promoting lazy
data, now applies to all variables after revision 1.75.  (This isn't
the only spot it gets used now, the tainting changes made it affect
form values, cookies, tainted form values, etc.)  It would increase the
speed at which variables are found--but I'm not sure its really an
intended side effect, and now I'll explain how I was bitten by it.

Its not unusual to see people write methods to be published that are
similar too:

 manage_main = DTMLFile("edit", globals())
 def manage_changeFoo(self, REQUEST, something=None, or_other=628):
    # stuff
    return self.manage_main(self, REQUEST)

They're a dime a dozen, yay neat.  What sometimes makes them more
interesting is when people modify the REQUEST dictionaries to pull
off various behind the scenes fake-like-we-requested-it-that-way
stuff.  This is what several of the methods that handled the Taller,
Wider, Narrower, etc. buttons on text area prefs did, and thats why I
ran into it.  One of these methods did:
REQUEST.form.update({'something':'x', 'or_other':'y'})
and then it returned a DTMLFile plied with the snazzy new request that
had been tricked out with these fake values now stuck into the form
dictionary.  This explains the TALES I saw that was
"request/form/something | request/something | default" which when I
first saw made wonder if the author was on acid.  It was clear to me
that 'something' would never be in other, it wasn't a special variable
and it wasn't being explicitly stuck into the other dictionary by the
code, so why the redundant TALES I wondered?  I removed the first
statement, and it broke the functionality.  Hmmm!

Until about 20 minutes ago I didn't understand exactly how object
methods got published, but I tracked down the code in Publish.py and
found the mapply() function and now its all clear to me what was
happening.  The request object gets mapply'd to the published method,
this means that any positional arguments or any keyword arguments will
get HTTPRequest.get'd out of the request object when they are applied
to the published method, and thanks to the side-effects from revision
1.75 on, it also means any named variables in a method definition will
be promoted into the HTTPRequest.other dictionary.  Now, let me just say
- if thats how its supposed to be, so be it - but spin my nipple nuts
and call me Frank, this does NOT strike me as terribly intuitive
behavior.

Anyway, it was a learning experience for me, but I'm not convinced
this isn't a bug.  What do you think?

(the patches I spoke of should be ready sometime tomorrowish assuming
I don't run into any other funkyness, I'll post them to the collector)

-- 
Jamie Heilman                   http://audible.transient.net/~jamie/
"We must be born with an intuition of mortality.  Before we know the words
 for it, before we know there are words, out we come bloodied and squalling
 with the knowledge that for all the compasses in the world, there's only
 one direction, and time is its only measure."		-Rosencrantz