[Grok-dev] Re: Developer scenario

[David Niergarth]

Martin Aspeli <optilude at gmx.net> wrote: 

> I would argue that your structure is not appropriate. :)
> 
> Grok is about developing things on the filesystem; most modern Zope 2 
> development (e.g. Plone) and to my knowledge all serious Zope 3 
> development follows that pattern as well. The ZODB is not a great place 
> to manage code. It is a great place to manage content and persistent, 
> hierarchical, semi-structured data.

Our structure developed organically using Zope 2 over the last four to five years. We did a good job of not mixing code and content but ended up taking out  the content and leaving the code, unlike most, mainly because our app is RDBMS-centered. This has actually worked exceptionally well for rapid application development. In this mode, we're really just using Zope 2 as an application server. If we could just get the code on the file system and keep using the Zope app server infrastructure we'd be very happy. Our app is quite complex (>1,000 scripts, page templates, zsqls) and our page templates use tal extensively so being able to reuse the pieces would be a big help. 


> If you don't need that, then Grok's patterns could help you, but it 
> doesn't really have a strongly formulated RDBMS story yet. You may 
> indeed find that it's easier to look at something like Pylons or Django 
> or TurboGears, which are much more centreed around RDBMS's.

Yes, that's probably where we'll head, rather than trying to productize everything in Zope 2.


> No. You cannot acquire things like that, from one module to another. 
> Arguably, there are better ways of achieving this kind of generality, 
> using the Component Architecture. You'd need to learn a bit more about 
> utilities and adapters before you could make that call, though.
> 
> > * Is there a way to handle ZSQL? (I've not been able to find any
> > examples.)
> 
> I don't think so ...

Is anybody else able to speak authoritatively on ZSQL in Zope 3?


> > * Is the psycopg2 database adapter accessible from within grok?
> 
> ... and probably not, if it's a zope 2 thing. However, if you need to 
> talk to relational databases, SQLAlchemy is a really great library. It's 
> not Zope specific (which is a good thing!), but there are some Zope 3 
> integration packages in svn.zope.org.

I did find some mentions of people using the psycopg2 database adapter successfully with Zope 3 -- but overall, very little detail.

We took a thorough look at SQLAlchemy recently but discovered scaling problems we could not solve. The Python profiler showed a lot of baggage being added to allow arbitrary updates that we rarely need. Using psycopg2 directly proved to be orders of magnitude faster for many of our queries. But our databases are large (millions of records, tens of GB in size), so we're not typical there either.

> It sounds to me like you wouldn't be "migrating" your application if you 
> moved it to Zope 3 or Grok - you're rather be re-writing it. You may 
> find that the added flexibility, manageability, testability (did anyone 
> ever write a unit test for an in-ZODB python script?) and power of Zope 
> 3, maybe with Grok to pull it all together, would pay off.

No doubt, and very appealing. The biggest problem sounds like our nested structure. The nesting creates the URL structure for a site, with page templates at one level redirecting to page templates at another level. Each level keeps related parts grouped together -- except for shared scripts, which are collected together higher up the hierarchy and so accessible via acquisition (e.g., context.tools.some_common_tool() ).

If anyone else has any thoughts, I'd appreciate your input. At this point it would probably make sense to take my questions to the Zope 3 lists.

> Martin

Thanks, Martin, for your thoughtful comments!

David Niergarth