Zope + Ape + Subversion (was: RE: [Zope-dev] Using a truely r evis ion based storage for Zope ?)

Tue Apr 13 20:53:30 EDT 2004

Hello,

Hmmm, well it's as stable as Ape and Subversion are respectively :)

I wouldn't call it stable no, it's something I did over the long week-end we
just had, and that's about it :)

Ape is at 0.8 and therefore becoming quite mature, I'd have to let others
speak as to it stability however ...

Subversion is also probably quite stable (It just reached 1.0), though I
don't know how heavily tested it's been in a long running process (Might it
have some memeory leaks ?) ...

Basically, I wouldn't recommend for usage yet.  I'm not even distributing it
just yet, mostly simply because right now it's code is still intermingled
with Ape's ... As soon as I pull it out into a seperate product I'll be sure
ot post it on zope.org and notify the list, as I'm sure I'll be looking for
feedback/testers.

I would estimate that moment to be weeks away though ...

Thanks,
J.F.

-----Original Message-----
From: Arthur Chan Chi Chuen
To: Kapil Thangavelu; Jean-Francois.Doyon at CCRS.NRCan.gc.ca
Cc: zope-dev-bounces at zope.org; shane at zope.com; zope-dev at zope.org
Sent: 13/04/2004 2:13 PM
Subject: Re: Zope + Ape + Subversion (was: RE: [Zope-dev] Using a truely
revis ion based storage for Zope ?)

Hi all,

I've read your discussion about version control, it seems a cool thing
you 
guys making good progress. Btw, can I ask is the Ape using Subversion in
Zope 
stable? how able CMF stuff? I wanna make/find a document management
system 
which can provide some kinda version control in Plone.

Thanks
Arthur
On Tue, 13 Apr 2004 04:06:26 -0400, Kapil Thangavelu wrote
> On Mon, 2004-04-12 at 18:03, Jean-Francois.Doyon at CCRS.NRCan.gc.ca
wrote:
> > G'Day,
> > 
> > Well, step one is done ... I now have Zope + Ape using Subversion as
it's
> > "filesystem" !!
> >
> 
> cool!
> 
> > This is step one because, as Shawn suggested (Thanks for the
pointer, 
that's
> > what I needed!), this simply means that Zope uses SVN purely as a
> > filesystem.
> >
> 
> > Because of subversion's nature, I want to look at 2 things beyond
this 
that
> > traditional filesystems don't support:
> > 
> > - Use zope's username for SVN logging.
> 
> using AccessControl.getSecurityManager().getUser().getId() and 
> setting it up as a revision prop ( or directly when creating the 
> repo transaction) should do it.
> 
> > - History/Undo support: View past revisions of an object, and revert
to 
such
> > a past revision.
> 
> perhaps the code for this would be helpful, (remove image for code 
> link) 
http://zope.org/Members/k_vertigo/Products/CMFSubversionBrowser/FileRevi
sions.
png
> 
> > - Zope Version support: SVN is fully transactional and atomic, this
should
> > allow for support of Zope versions (I think ?)
> >
> 
> zope version support isn't really all that worthwhile, esp in a cmf
> context. in general zope's version support (or zodb more 
> particularly) is a database level feature masquerading as an 
> application one. since objects modified in a version are in essence 
> locked from participating in other transactions, actions like 
> modifying content in a version in a cmf site amounts to locking the 
> catalog from changes outside of the version, which amounts to 
> shutting down write activities to a cmf site. otoh, integration with 
> zope's transaction manager would be a good thing, although its 
> problematic to integrate between svn and zope txn models, more on 
> that in a moment.
> 
> > In the longer term, there's great opportunity for:
> > 
> > - "Built-in" conflict management and resolution: No more need for a
> > "SafetyBelt" type approach.  Right now I haven't looked at this at
all.  I
> > plan to implement smart merging where possible (It might work
already
> > actually, I just need to test it).  True conflicts (Where a merge
can't be
> > accomplished withouth user interaction) would raise some sort of
conflict
> > error.
> >
> 
> i don't know that conflict management is really useful in this
context.
> svn like zope relies on optimistic concurrency control, and currently
> doesn't support dav locking (which zope does). ie, it will just 
> throw an error if the content has been changed since the transaction 
> began. the 'normal' concurrency control of svn is better but 
> dependent on using the working copy (wc) layer, which is additional 
> programming and storage overhead. so at the layer of the svn client 
> this is already done and working and good, but integrating this 
> functionality into zope is a bit harder without wc overhard.
> 
> this also makes the transaction integration becomes harder because
both
> zope and svn are using what amounts to optimistic concurrency control
> which makes it impossible afaics, to get real txn integration, ie in
> zope's two phase commit strategy, the last chance for a participant 
> to safely abort is tpc_vote, but there is no real way of knowing if 
> the svn txn will suceed or not until its tried. if it is tried at 
> this stage and succeeds then there is the possibility of a latter 
> txn participant failing the tpc_vote and the txn being aborted, and 
> if waits till tpc_finish (last part of two phase commit) and the svn 
> txn fails it can hose the composite txn integrity.
> 
> once svn supports dav locks, doing txn integration via resource
locking
> as part of tpc_vote (or earlier) would be possible, till then.. i 
> dunno, i can't see a way around this for real txn integration.
> 
> i'm also curious how you dealt with svn transactions as part of the
ape
> integration work to date.
> 
> > - Editing Zope content objects through interaction with the svn 
repository.
> > I can checkout the repository, edit some objects, and chek them back
in,
> > never interacting with Zope directly ... I've already tried this !
Works
> > great for text based content types such as PageTemplates or DTML
Documents
> > and so on ... I even did it with a JPG, though because the
properties hold
> > width and height, you get some weird looking pictures :) The concept
is
> > valid though.  There may someday be a way to leverage this
functionality
> > better with a purpose built client of some sort.
> 
> to me this is one of the fundamental benefits of using svn, giving
users
> the ability to use interfaces like tortoisesvn (win shell extensions)
>  or mac finder extensions to interact directly with content.
> 
> > 
> > - Leveraging SVN's property management.  Content in SVN has
properties, 
much
> > like Zope does.  I haven't looked at it yet, but I've 
noticed ".properties"
> > file appearing ... I'm guessing those are the Zope properties, which
would
> > be better handled by subversion's property mechanism.  And
properties are
> > versioned too !
> 
> definitely!
> 
> > 
> > In the realm of the wishful thinking, there's even more:
> > 
> > Right now, HEAD (Latest/youngest revision) is always used and worked
with.
> > The really powerful feature I want to eventually get to is
publsihing
> > something of a given revision, while editing another.  One potential
> > paradigm for distiguishing between the two modes of operation could
be to
> > use anonymous vs. authenticated.  This is not useful to everyone,
but can 
be
> > in certain circumstances, most notably where authenticated =
> > authors/developpers and anonymous = normal users.  This however
requires 
ZMI
> > interfaces, and in my case CMF ones as well ... This would be global

though
> > ... Eventually it'd be nice to have per object control of this
stuff.  
Andy
> > McKay says it can't be done, anybody care to contradict him ? :P I
image 
I'd
> > have to monkey pathc something DEEP in the Zope code base, but I
find the
> > mix-in class that's the commonn denominator ... why not ?
> >
> 
> to me this seems a separate point then svn usage at all, its really
> about content view paradigms for revision workflow. the two
predominant
> ones are using staging to deploy different sites representing
different
> parts of the publication cycle/wf, and the other which afaik, is only
> widely used by silva is view multiplexing where the view of a 
> content is responsible for retrieving the proper version to render a 
> display for, although afaicr silva support for such is initimately 
> tied to its workflow and versioning paradigm.
> 
> > Anyways, I'm just rambling by now ... Comments, thoughts and
constructive
> > criticism welcome !
> 
> i've done some thinking and work with integration of svn as content 
> in zope as well, though using a different software stack than ape.
some
> notes/ideas culled from a dev diary i kept while working on it that
> might be useful.
> 
> - using svn hook scripts for event notification back into zope
(content
> reindex), although it should be careful about modifying the content
> otherwise some client which just committed might be put out of date.
> 
> - transform layers (using portal transforms) with a private content
type
> registry... although integrating at the ape layer this might not be 
> as useful.
> 
> cheers,
> 
> -kapil
> 
> > 
> > Thanks,
> > J.F.
> > 
> > -----Original Message-----
> > From: zope-dev-bounces at zope.org [mailto:zope-dev-bounces at zope.org]On
> > Behalf Of Shane Hathaway
> > Sent: April 8, 2004 11:20 AM
> > To: Jean-Francois.Doyon at ccrs.nrcan.gc.ca
> > Cc: zope-dev at zope.org
> > Subject: Re: [Zope-dev] Using a truely revision based storage for
Zope ?
> > 
> > 
> > Jean-Francois.Doyon at CCRS.NRCan.gc.ca wrote:
> > > I've started looking at the ZODB and APE packages to try and get
some
> > > understanding of how the whole storage interaction works, but
it'll take
> > me
> > > some time to figure it all out ... So I thought I'd get feedback
on the
> > idea
> > > first ...
> > 
> > Sounds great!  If I were you, I would start by replacing 
> > Ape/lib/apelib/fs/fileops.py with something that uses the Subversion

> > module.  It's a very simple module and you should be able to get
pretty 
> > far that way.
> > 
> > Shane
> 
> _______________________________________________
> Zope-Dev maillist  -  Zope-Dev at zope.org
> http://mail.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists - 
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope )

--
Open WebMail Project (http://openwebmail.org)