[Zope] ZoDB and BIG FILES

[Sebastien Pastor]

Thanks Paul for all those infos.
Would storing files using MySql or  PostGreSQL be another viable 
solution? Have you tried that ?


Seb

Paul Winkler wrote:

>On Tue, Jan 13, 2004 at 12:01:09AM +0100, Sebastien Pastor wrote:
>  
>
>>Hi there,
>>
>>
>>I am about to finalize a Zope 2 site. It will mainly play the role of 
>>groupware app and storage server.
>>My question is : i know i ll get quicly a big amount of PSD files 
>>(photoshop) they are usually quite heavy. Do you rekon i should store 
>>them on my server file system rather than the ZODB ? Do you see any 
>>trouble putting them all into the DB in a longer term  ? (i know my 
>>python is large-file compiled). 
>>    
>>
>
>two issues:
>
>1) A single large FileStorage has been a bit cumbersome to maintain
>(backups etc). recent developments (repozo.py) make it possible
>to do an incremental backup.
>There's also DirectoryStorage which on a suitable filesystem should 
>be able to scale to a very large database.
>You can also mix-and-match and have multiple databases using DBTab.
>
>2) Performance of serving large files or images from Zope is pretty
>bad, much slower than a static webserver like Apache. The worst
>part is that a few simultaneous requests for large files will
>essential DOS your entire zope site until they complete.
>
>I have not identified the precise point of pain, but I know that
>in my experience, 5 MB is OK and 50 MB is awful. I have worked
>around this by using FSCacheManager (available from CVS at
>collective.sf.net) which does "funky caching". It is easy to set 
>up with Apache, but is only safe to use with anonymously-viewable content.  
>I understand that Squid can be set up to do filesystem caching while 
>forcing authentication on each request so it would be safe with 
>secured content, so we might do that instead at a later date.
>
>  
>
>>Storing on both FS and ZODB is it an 
>>easy thing to do ?
>>    
>>
>
>If you want the files on the filesystem, there are various solutions
>depending on your needs: LocalFS, FilesystemDirectoryView, and APE. 
>I have not evaluated these for performance with large files, except
>LocalFS which I know is at least as slow as normal zope File objects.
>I tend to assume they are all slow.
>
>  
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.zope.org/pipermail/zope/attachments/20040113/f73f0af6/attachment.html