[Zope] should files be stored in RDBs?

[Charlie Fulton]

To address comments from several people:

> Andreas:
> Have you thought about storing the data inside the ZODB?

I had thought of it but wasn't sure if it would be appropriate.  Is it
possible to assign arbitrary metadata to a file in the ZODB?


> Tom P:
> It's certainly feasible to insert data into a relational database using
> DTML.  If the data is truly binary, it's probably not very well suited for
> web forms, where some characters are not allowed without escaping.

Wouldn't using <type=file> and <enctype="multipart/form-data"> get around
the need to escape certain things?  I'm looking to build something that is
simple to use and preferably wouldn't need any special applications for the
client.


> It also would depend on what you want to do with the data, and
> whether there
> is any metadata that needs to be attached to the data.  For example, it's
> one thing to store a blob in a table, and another to find the right data
> later on.  Of course, you have the same problem when you store files.  You
> may find that it's better to store the files in the file system and the
> metadata in a database, either the ZODB or in a relational database.

There will be metadata (what equipment was used, what settings were, who did
it, etc) which needs to be searchable and is appropriate for a relational
database.  Now what to do with the data itself?  File names are similar and
I'd probably have to hash the timestamp or something in order to guarantee
that they're not the same in a directory, making it essentially meaningless
without the database.  Plus they're small and the hierarchy could get ugly
in a hurry.

So I thought I might go ahead and store them as records in the database.  It
seemed simple in principal and more intuitive since whoever inherits this
system will likely not be a professional IT person.  Just as I am not one.



> I'm setting up a database to store data file for a scientific research
> group.  I'm thinking of writing the data into a MySQL table rather than
> storing them externally in the filesystem.  The files are generally small
> but the number of them will grow quickly.
>
> Are there opinions as to whether this is a good or bad idea?
>
> I know there's overhead and a (severe?) throughput penalty, but I was
> thinking this would be simple to manage (and back up) as well as being
> easy
> to search.  The load on the server won't be much anyway.
>
> Can collecting binary data in a web form and writing it to MySQL be done
> totally in DTML?  I've found some past threads in the list dealing with
> this
> kind of thing (and an old how-to) but there doesn't seem to be a consensus
> on a "right" way to implement it.