[Zope-dev] Experiments with ORMapping

[Phillip J. Eby]

At 11:01 AM 5/11/01 -0400, Shane Hathaway wrote:
>Joachim Werner wrote:
> >
> > The current design plans of SmartObjects are mainly based on the assumption
> > that we will not be able to change Zope itself. This is not a dogma for us,
> > however. I guess doing OR-mapping in the Zope core would be fine with 
> us ;-)
>
>FYI by replacing I don't mean changing anything in the Zope core.  I
>mean using a different class in place of Connection, which you can do
>just by creating a custom_zodb.py.

Hm.  So you're suggesting creation of a Storage class that returns a 
special root object which emulates the standard ZODB root 
PersistentMapping, and contains another object that emulates a folder, with 
a bunch of other foldoids that are actually tables, or something of that 
sort?  And maybe exposes some query methods ala ZCatalog?

I'm not quite clear on how exactly you suggest mapping from RDMBS -> 
ZODB.  There's a *significant* (IMHO) impedance mismatch between ZODB's 
arbitrarily identified variably structured single records and SQL's 
content-identified fixed-structure record sets.  This is what application 
frameworks/toolkits (such as your own DBAPI product) are needed for.


> > But the more important one for us is that we still believe that even with
> > ZCatalog Zope can not really do efficient "croos-tree" queries in all 
> cases.
> > A query like "Give me all users who have bought this and that product and
> > are aged 20 or above" can not be handled by the catalog, I think.
>
>Yes it can. :-)  That's not to say that it has as much flexibility as a
>SQL query, but it can do most of the things people usually do with one
>table at a time.
>
>Here's the goal I envision for OR mapping: to be able to move between
>OODBMS and RDBMS seamlessly.  It makes sense to develop on top of ZODB
>then move to an RDBMS for testing and deployment.

Unfortunately, I think that this requirement can *only* be met through a 
common API or access pattern/framework/what-have-you, be it DBAPI, 
ZPatterns, SmartObjects, or TransWarp.  The ZODB is both "too powerful" (in 
its flexibility) and "too weak" (in lack of any ZODB-level notions of 
record sets, schemas, and indexing) to be useful as a 
cross-database-platform API.  That's not to denigrate any of the value of 
ZODB itself - an explicit goal of both ZP and TW is to leverage ZODB's 
flexibility in combination with other kinds of databases.  (And, recently, 
I have been speculatively eyeing the ZODB for some mortgage-industry 
related projects which involve complex variable data structures, 
distribution requirements ala ZEO, and local data stores on 
intermittently-connected laptops.)



> > SmartObjects is more of a programming framework than just adding OR-Mapping
> > to Zope. So if we can solve the storage and query parts more efficiently by
> > just having Zope itself extended a bit, this would be very cool ...
>
>You see, I think it is not necessary to create a programming framework
>if the goal is OR mapping.  The framework is already defined, and it's
>Python / ZODB.  But SmartObjects seems to have many loosely related
>goals, making it difficult to assist.


I think the goal for SO, and all the other frameworks that have been 
created or are being created, is to *make application development 
easier*.  OR mapping is just a means to that end.  LDAP connectivity, or 
other types of non-ZODB data access are important to many of us, as 
well.  This boils down to abstraction of how "data" attributes are 
represented.  For example, if I am an ISP, and I want to implement an 
"active" flag on an "account" object, I would like changing it to 
automatically go out and add or remove routing entries and password entries 
on my servers when I update the record through my Zope interface (web, 
SOAP, or whatever).  Could you make a ZODB "Storage" object that supported 
this?  Maybe.  But from an architectural standpoint it would be rather 
messy - akin to writing OO code with giant "switch" statements.

The Java (and CORBA, actually) solution to this, is to use a "property" 
pattern, where a property is a non-real thing that actually only has 
setters and getters.  An object's users always go through these accessor 
methods, so the implementation can be anything you like.  (Notice, btw, how 
the solution is once again a framework pattern...)

ZPatterns was an effort to produce a Pythonic emulation of this concept 
that didn't need actual methods.  It unfortunately inherits some of the 
flaws of __getattr__ hooks, such as forced semi-centralization of 
implementations. TransWarp does away with that and instead introduces 
property ("feature") objects which themselves have methods, but are not 
data.  This provides a bit more extensibility than the Java approach, since 
it is conceivable one could add an "observable" interface to a property 
object and thus subscribe to it, for example.

Anyway, all I'm really trying to say is, if you want implementation 
independence, you have to have implementation hiding.  ZODB makes it easy 
to store Python objects with hardly a second thought.  However, if you plan 
to use something other than ZODB for *any* part of your object at any time 
in future, you need an API that hides the implementation.

Now, if y'all want to create a "ZODB 4" that can load portions of an 
object's state from different backends, *that* would be cool.  But I think 
you'd find that it would impose a penalty for the mapping metadata, and the 
associated complexity that would bring to the ZODB "kernel" level.  At 
least the app frameworks (including DBAPI) put this metadata at the 
*application* level, where it more visible and controllable by the 
user.  And this metadata really is application data, anyhow, since it is 
all about how the application wants to see the data, not how the data 
"really is" in physical storage.