oodb philosophics ;) was: Re: [Zope-dev] Experiments with ORMapping

Ken Manheimer klm@digicool.com
Fri, 11 May 2001 10:41:21 -0400 (EDT)


On friday, 11 May, Joachim Warner wrote:

> > The other motivations for an RDBMS are (1) people have existing schemas
> > and want Zope to access the same data as their existing apps, and they
> > want it to be transparent, and (2) tables with millions of entries are
> > easily stored in Zope but the perception is that the catalog isn't as
> > fast as a database index.  No one has done any tests AFAIK.
> 
> Then we should do these tests. E.g. I'd like to see:
> 
> - 20 GB of Word and PDF documents stored in the ZODB and full-text +
> metadata indexed in ZCatalog
> - the complete zope@zope.org mailing list archive in the ZODB
> 
> If Zope can handle those without the help of external tools (RDBMS etc.),
> I'll use it for all our Document Management ...

As a matter of fact, we did a quick CMF demo that has the content of the
zope list, zope-dev, and many of the other zope.org lists, and the
comp.lang.python list for the past few years.  The catalog searches are
very very fast, i can't recall if the demo was set up with some
interesting canned CMF topics, but the things works well.

The picture isn't altogether rosy - the process of loading the objects was
arduous.  On the other hand, the exercise (actually, a subsequent one with
simpler article objects) served as the basis for tuning the cataloging
process, and may have helped it get a lot better.

If i have time next week, i'll see if we have the corpus online somewhere.  
(The lists were complete up to a few months ago.)  Eventually we'd like to
be incrementally stuffing new messages into the database as they arrive.

The catalog has required some substantial work investment, but from my
viewpoint (particularly since i haven't had to work on it!-), it's well
worth it.

> > Actually OracleStorage and bsddbstorage, recently released, are designed
> > to address concerns about performance and reliability, and they do an
> > excellent job at it.  And I consider ZODB as "real" an OODB as anything
> > else.  (In some ways it's the best out there IMHO.)
> 
> Agreed. ZODB has a much longer proven history of success than most other
> OODBs.

It is quite useful!

I have no experience programming with other object databases, and very
little with relational databases, so i have no basis for comparison.  
What i do know, as a python and zope programmer, is that ZODB is
spectacularly useful as a persistent store - as flexible *and* as powerful
as i could want.  The addition of ZEO manages to significantly increase
that usefulness!  The work we/pythonlabs (and andrew kuchling, etc) is
doing to enable use of it as an independent entity can only help improve
it's usefulness for everyone.

Ken Manheimer
klm@digicool.com