[Zope-dev] ZPT Optimization Opportunity

Tres Seaver tseaver at zope.com
Mon Sep 27 14:18:30 EDT 2004

Fred Drake wrote:
> On Mon, 27 Sep 2004 11:25:16 -0400, Tres Seaver <tseaver at zope.com> wrote:
>>Yes, you read it correctly (although I think the real answer is
>>"identical performance, within margin of error").
> With other benefits, that's good enough, too.
>>>Is there anything to indicate that memory usage is actually improved?
>>I'm not there yet.  This is essentially a "zero-copy" optimization;  if
>>I can show that the functionality is equivalent, with approximately the
>>same performance under unloaded sites, *then* I can look into whether it
> Ok; understood.
>>I envision two possible benefits, which will materialize only "at scale":
>>   - Removing the need for the appserver thread to malloc and copy the
>>     Big String *may* be a win on a memory-constrained system (large
>>     allocations can induce funny non-linearities on the underlying
>>     malloc implementation).  Not having to copy values, even in C,
>>     should be a "pure" win, in any case.
> Agreed.  Doing less is always a win, eventually.  Often in combination
> with doing just a little less in lots of places, but that's ok.
>>   - Returning the iterator to medusa means that the application
>>     thread becomes available to service other threads that much more
>>     quickly.  Even if the malloc issues can't be demonstrated, the
>>     increase in concurrency *should* be a win here.
> This is interesting.  If you're pushing content out the socket
> earlier,

I'm not looking for that;  only to avoid constructing the big result 
string just to hand off to the publisher.

  you need to guard against failure much earlier, or be willing
> to deal with it in different ways.  This is likely less of an issue
> for sites that have been well-tested and are considered ready for
> production.
> I'll need to review the patch more carefully, especially since I don't
> know much about the Medusa layer.  Does the modified code deal
> properly with errors encountered late in the transformation of a
> template?

Transformation is already complete at that point.  The only difference 
is the type of the result returned (eventually) to the publisher.

BTW, I looked again at where StringIO is used, and it seems that ZPT 
constructs extra StringIO objects for (at least some) nested blocks.  In 
that case, we might be able to extend the win by having the "calling" 
bit use 'list.extend' for its own buflist, instead of calling 'getvalue'.

Tres Seaver                                tseaver at zope.com
Zope Corporation      "Zope Dealers"       http://www.zope.com

More information about the Zope-Dev mailing list