[Zope-dev] Re: [Zope3-dev] proposal: serving static content faster

[Paul Winkler]

On Thu, Apr 08, 2004 at 08:02:05PM -0400, Shane Hathaway wrote:
> On 04/08/04 12:16, Paul Winkler wrote:
> >On Wed, Mar 24, 2004 at 01:32:18PM -0500, Shane Hathaway wrote:
> >
> >>Jeremy has suggested that object pre-fetching could be added to ZODB.
> >
> >
> >This is much on my mind currently.
> >Any thoughts on what an API for pre-fetching might look like?
> 
> Well, thinking about it some more, it seems like it might be just as 
> easy to run a special thread just for fetching blobs.  One queue should 
> tell the blob thread what to get, and another queue should return the 
> results.  I think that would be a nice design.

ok... i'll mull that over.
 
> >The use case that most concerns me is:
> >If you have an Image or File object with a very long Pdata chain,
> >you're likely to A) thrash the ZEO client cache if it's not big enough,
> >and B) spend a looooong time waiting for all the objects in the chain
> >to load.  At least, this is my theory on what's slowing things
> >down - I will be trying to verify this today. See the bottom of this page:
> >http://www.slinkp.com/code/zopestuff/blobnotes
> 
> In fact, Zope puts large files (the threshold is around 256K - 512K) 
> into a temporary file before serving them, to free up application 
> threads.  It's a tremendous handicap.

If you're referring to the code in ZServerHTTPResponse.write:
AFAICT, since Image.File calls RESPONSE.write() for each Pdata
in its chain, we'll actually get a temporary file for each Pdata 
rather than one big one for the whole file.
There may be (probably are) performance issues in there, but
right now I'd be really happy to get ClientStorage a bit closer to the 
speed of FileStorage for this Pdata gunk.

-- 

Paul Winkler
http://www.slinkp.com