Chris McDonough wrote:

This is already done on the zodb-blobs-branch. I would be happy to create a mountpoint-branch that does not externally link the ZODB with blob support, then merge the changes in to the Zope 2 HEAD (and 2.9 branch if one exists).

Ah, I had forgotten about that. It would be great to merge the mountpoint work into the head. There isn't a 2.9 branch afaik.

November 1 feature freeze, eh?

Yup. :)

I'd love to get blobs in before this too but I don't know if it will be possible.

I don't think so. Nov 1 is less than 2 weeks away. I think we need to be realistic.

I would definately want to review this before it got merged into the head. I think Tim might too. I don't think it's ready for my review yet. Anyway, if you want to make this a priority, I'm willing to make time for review as best I can. If you want to try to finish this before Nov 1, I'll try to help by doing timely reviews.

Note that this will be of limited usefulness without also updating Zope file implementations (z2 and z3) to take advantage of it, although people could build add-on packages that used it.

If not, it would really suck.

Well, I dunno, if it's not ready, June is not that far away. I'm confident that we could get this done by the May 1 feature freeze if this was a priority.

Jim

[Chris McDonough]

...

...

This is already done on the zodb-blobs-branch. I would be happy to create a mountpoint-branch that does not externally link the ZODB with blob support, then merge the changes in to the Zope 2 HEAD (and 2.9 branch if one exists).

[Jim Fulton]

...

Ah, I had forgotten about that. It would be great to merge the mountpoint work into the head. There isn't a 2.9 branch afaik.

FYI, Zope trunk (in SVN) is current 2.9 development.

[Chris]

OK, I will do so.

In that case, I'm willing to share <wink> this set of probably-related open Collector issues:

ZODBMountPoint should not monkey-patch ZODB http://www.zope.org/Collectors/Zope/1525

Clean up mounts, TemporaryFolder http://www.zope.org/Collectors/Zope/1800

ZODB: Mounting broken for non default transaction manager http://www.zope.org/Collectors/Zope/1875

[It doesn't look like my response went to the zope-dev list; re-sending.]

On Tuesday 18 October 2005 15:43, Tim Peters wrote:

I'm copying Fred because he may remember more about this than I do. Fred, do you know of a reason why I can't stitch a newer ZODB into Zope(2) trunk? I have a dim, fading memory of the last attempt failing, and of agreeing in email to wait for the "5 guys" to "do something" before trying again. Sorry for not being more specific ...

We need to do the zpkg/ZODB switch all at once because it affects how extensions get their include files. When I tried switching the Zope 2 trunk before, there was a problem due to Five tests failing. I don't remember the details, but the Five-folks seemed to think things would be better with newer versions of Five.

Since Five development is done elsewhere, though, it's hard to tell what the deal is with that. I snapshotted what I did get done as the zpkg-build-branch, so we can get back to it without duplicating work.

-Fred

-- Fred L. Drake, Jr. <fdrake at gmail.com> "Society attacks early, when the individual is helpless." --B.F. Skinner

On Tue, 2005-10-18 at 11:32 -0400, Jim Fulton wrote:

Ah, I had forgotten about that. It would be great to merge the mountpoint work into the head. There isn't a 2.9 branch afaik.

I think I may need some remedial SVN help because I don't want to do this in a stupid way. Hopefully someone will be willing to guide me through this.

There exists a branch of the Zope 2 trunk named the zodb-blobs-branch. It was created as a branch of the Zope 2 trunk on September 25 (about 3 weeks ago).

The zodb-blobs-branch contains code that use Tim's multidatabase support for mountpoint support rather than the older DBTab code which monkeypatched ZODB and did other nasty things. It contains a few other ancillary changes that make it possible to use a HEAD-ish ZODB package in Zope 2 as well (including changes to the setup.py I mentioned which allows a newer ZODB to be compiled).

The zodb-blobs-branch happens to link in an svn external for the ZODB package which currently points to a *ZODB* branch named "blob-merge-branch". This is really the only thing "blob-ish" about the zodb-blobs-branch. All of the changes that actually cause blobs to be supported are isolated in that ZODB branch. The rest of the changes are really just to support a later ZODB revision within Zope. It's likely that this svn external can be changed to point to any 3.6-ish ZODB branch without breaking things too badly.

So I'd *think* I'd like to:

- create a SVN branch *from the zodb-blobs-branch* named mountpoint-merge-branch

- change the svn external for lib/python/ZODB on the mountpoint-merge-branch to point at the proper ZODB revision

- merge the changes that have happened since September 25 on the Zope 2 trunk into the mountpoint-merge-branch

- merge the mountpoint-merge-branch back in to the trunk.

But I've read the Zope SVN FAQ and it frowns on the practice of merging trunk changes into a branch and back again. Is there a better way to do this other than applying the changes manually to the HEAD?

Also, what is the "right" branch of ZODB to use in the svn:external for lib/python/ZODB so I can test that it all works ok before I actually perform the merge to the HEAD?

- C

On Tue, 2005-10-18 at 22:21 -0400, Tim Peters wrote:

[Chris McDonough]

...

I think I may need some remedial SVN help because I don't want to do this in a stupid way. Hopefully someone will be willing to guide me through this.

I'll be in FB tomorrow if you'd like to pair on it (while in theory Jim might object, I think he thinks getting this done is important enough to offer real help -- especially if he doesn't have to offer it himself <wink>).

Thanks for the offer! I won't be able to visit ZC world HQ tomorrow, though unless you'd be there and willing to start around 10pm. "Other duties" is my official excuse but I'm also horrified by the idea that I'd be expected to wear pants if I came over there. That's just uncivilized. :-)

...

The zodb-blobs-branch contains code that use Tim's multidatabase support for mountpoint support rather than the older DBTab code which monkeypatched ZODB and did other nasty things. It contains a few other ancillary changes that make it possible to use a HEAD-ish ZODB package in Zope 2 as well (including changes to the setup.py I mentioned which allows a newer ZODB to be compiled).

Check. Question: does zodb-blobs-branch contain anything you _don't_ want to see on Zope trunk now? You didn't mention anything like that here.

No (save for inappropriate svn:externals to ZODB and ZEO).

...

• merge the changes that have happened since September 25 on the

Zope 2 trunk into the mountpoint-merge-branch

Why? It may create headaches and I don't see the attraction. Have people been checking in changes to the same files over the last 3 weeks? Even if they have, conflicts are probably easier to deal with when the new branch gets merged back to the trunk.

No, people haven't been changing the same files (except for maybe setup.py) so seems like good advice. This is really what I needed to understand.

...

But I've read the Zope SVN FAQ and it frowns on the practice of merging trunk changes into a branch and back again.

That's because it's so easy to lose track of what you're doing then. It probably can't be avoided on very long-lived branches, but this branch is only several weeks old now, right?

Yes. The zodb-blobs-branch can just die after this merge if there's a way to get delete branches entirely. If there is to be a long-lived branch, it will be the "blob-merge-branch" of ZODB.

Given that Zope 2.9 is not going to ship with blob support due to feature freeze, I think this means that we have until May to allow the blob-merge-branch to get utterly out of sync with the ZODB trunk. We can then easily wait until, say, the last week in April to worry about issues caused by that desynchronization. The work necessary to remerge should provide just the appropriate amount of delay to allow blobs to miss the next major Zope release. ;-)

...

Also, what is the "right" branch of ZODB to use in the svn:external for lib/python/ZODB so I can test that it all works ok before I actually perform the merge to the HEAD?

I never leave a Zope pointing at a ZODB branch -- only at a ZODB tag. The only two suitable tags at this time are

ZODB/tags/3.5.1

and ZODB/tags/3.6.0a4

Either should work fine for you. If you use the latter, it may save me some time later ;-) But in either case, it won't last long (I'll have to stitch in a 3.5.2 beta or 3.6.0 beta soon anyway, and tags for those don't exist now).

Great, that's what I needed to know.

Thanks Tim!

- C

[Chris McDonough]

There is a wrinkle about performing this merge that eluded my memory until now.

To support multidatabases within Zope, it was reasonable to change ZODB.config.ZODBDatabase to support the heretofore likely-unused-by-real-world-code "databases" and "database_name" options that may now be passed into ZODB.DB's constructor:

http://svn.zope.org/ZODB/branches/blob-merge-branch/src/ZODB/config.py?rev=3...

The current blob-merge-branch code depends on this change being available in the ZODB revision it uses. In case you're interested, the code that actually makes use of this feature in the zodb-blobs-branch is in the Zope2.datatypes.DBTab.getDatabase method.

Is this change acceptable for a merge into the ZODB HEAD?

Turns out that a release of Zope3 has already been made that supports multidatabases, and I'd naturally prefer to follow the lead of a Zope that's already out there. Jim showed me the Zope3 implementation code and an example today. I found the code easily (on Zope3 trunk), but can't for the life of me find anything there that looks like his example. Jim, where is that?

The Zope3 code in question is in

src/zope/app/appsetup/appsetup.py

function multi_database(). Note that they didn't change any ZODB files, instead they give values to a DB's .databases and .database_name attributes after constructing the DB. While that might be questionable in general <cough>, the implementation of multidatabases was meant to be both concrete and public. It's not an accident that ZODB's tutorial tests/multidb.txt doctest explains and exploits details of the concrete implementation -- it's not meant to be abstract. IOW, poking in new values for these attributes isn't considered to be evil.

I believe (here's where the example I can't find would nail it) they use the name on a <zodb> section as the DB's database_name. Fred points out that ZConfig section names are case-insensitive, forced to lowercase, so that

<zodb CHRIS>

and

<zodb cHris>

have the same name. That's not ideal, and threading these attributes throughout ZODB's config.py instead (as you did) would be a sane way to worm around that.

But for right now, I think doing it differently than Zope3 does it would cause needless confusion more than it would help. Enhancing Zope3 and Zope 2.9 in the same way(s) here could make sense.

Some mechanics: if we do need to make changes, ya, ZODB trunk is the place to do it. Work on the branch should use ZODB trunk now. When that's as ready to go as it's going to get, let me know and I'll make "an internal release" of ZODB 3.6 so we can use a ZODB tag before merging into Zope trunk. ("An internal release" just means I update ZODB's NEWS.txt, fiddle version numbers and dates in umpteen places, and make a tag so other projects can use that -- it's the tag that's important here; an internal release does not involve making tarballs, Windows installers, announcements (etc), so it's much cheaper and goes much faster (minutes vs hours) than making a public release.)

[Tim Peters, on multidatabase support in Zope3]

...

... Jim showed me the Zope3 implementation code and an example today. I found the code easily (on Zope3 trunk), but can't for the life of me find anything there that looks like his example. Jim, where is that?

[Jim Fulton]

Do you mean an example of a zope.conf that uses it?

Yes, that's all -- just a concrete example. I could have guessed one (and did later ;-)), but guesses can be wrong.

From a customer engagement:

<zodb main> <filestorage> path $DATADIR/Data.fs </filestorage> </zodb> <zodb a> <filestorage> path $DATADIR/A.fs </filestorage> </zodb>

Thanks! That did the trick.

We decided to use the section names for the database names. This was to avoid changing ZODB. I'm not sure that that was a good idea.

Let's change it.

This approach has two disadvantages:

• Because section names are case insenstive, database names end up being lower case, whether we want them to be or not.

• It may not be obvious that the section name is also the database name.

It isn't obvious -- and unlike for other <zodb> keys, a user can't look in ZODB's component.xml now to find out anything about this usage. They can, e.g., look there to see that they can specify cache-size, that it's an integer, and that it defaults to 5000. They can also find helpful <description> sections for many keys. Using the section name is pure out-of-the-blue magic in contrast.

I'm really unsure about whether this is a disadvantage. I'm not sure if:

<zodb>
  name main
  <filestorage>
    path $DATADIR/Data.fs
  </filestorage>
</zodb>
<zodb>
  name a
  <filestorage>
    path $DATADIR/A.fs
  </filestorage>
</zodb>

is better or worse than the first version.

I think it's worse, but mostly because a key with name "name" is also an option in _related_ sections, but with unrelated meaning. For example, if you had a nested <zeoclient> section there it could also have specified a "name" key, which would have nothing to do with the <zodb> key named "name". Nesting options with the same name gets confusing quickly. OTOH, I would like the explicit key better if it had a different name, say

<zodb> multidb-name main <filestorage> path $DATADIR/Data.fs </filestorage> </zodb> <zodb> multidb-name a <filestorage> path $DATADIR/A.fs </filestorage> </zodb>

I'm inclined to think that any time you have sections of the same type, it is desireable to give them names, in which case we might be tempted to list the names twice.

Sounds orthogonal to me. If it's desirable that whenever multiple sections of the same type appear, they must be given names, that's fine, but the way to enforce or encourage that isn't to make all sections that may appear more than once give some _meaning_ to the section name. It was just expedient in this specific case, right?

... I'd be happy to plumb this through the factories open method. It would seem to me that we only need to be able to pass a databases argument.

Right.

The factory presumably knows it's own name. It could then pass the databases dict and the name to the DB constructor.

If I change the <zodb> config to say there's a new optional key for the multidatabase name (like the "multidb-name" key in the made-up example above), then the factory will have the same access to that as it has for other existing ZConfig-specified keys (like cache-size).

BTW, I think there's a related buglet in Zope3's multi_database():

name = factory.name or '' ... db.database_name = name

Defaulting to an empty string for "the name" is really a bit of abuse, since the documented default database_name for ZODB.DB is "unnamed", and I doubt Zope3 documented that it changed this default ;-). An explicit ZConfig key here would supply that correct default.

... I haven't looked at Chris's changes. I was pretty happy that the changes we made in Z3 were fairly localized and small.

Adding the optional new key to the <zodb> config, and threading the `databases` arg thru ZODB's config.py, are also small changes.

...

But for right now, I think doing it differently than Zope3 does it would cause needless confusion more than it would help. Enhancing Zope3 and Zope 2.9 in the same way(s) here could make sense.

OTOH, this feature has hardly been used in Z3. I added to ZODB because I had been meaning to for some time ad because we needed it for a customer. I don't think anyone else has used it, so I don't think there's much established pattern in Z3. Then again, I'm not sure, except for the case insentitivity issue that we didn't do it the best way. I'd much rather revisit the case insenstitivity of section names in ZConfig. I think that if ZConfig section names were case sensitive or at least case preserving, I'd be happy with the approach we took.

Note that if we add an explicit new key, the case issue goes away (for this specific new option) at once.

[Tim Peters]

...

I think it's worse, but mostly because a key with name "name" is also an option in _related_ sections, but with unrelated meaning. For example, if you had a nested <zeoclient> section there it could also have specified a "name" key, which would have nothing to do with the <zodb> key named "name". Nesting options with the same name gets confusing quickly. OTOH, I would like the explicit key better if it had a different name, say

<zodb>
  multidb-name main
  <filestorage>
    path $DATADIR/Data.fs
  </filestorage>
</zodb>
<zodb>
  multidb-name a
  <filestorage>
    path $DATADIR/A.fs
  </filestorage>
</zodb>

[Florent Guillaume]

Yes, please. There is already confusion for cache-size, let's not repeat that with another key. Note that "database-name" is more expressive, I think

Since the name of the corresponding DB argument is "database_name", and all the docs that exist for this call it "database_name" too, that's hard to argue against ;-)

(the "multi" seems like an implementation detail to me).

Not really: a DB's database_name was introduced specifically for the new-in-ZODB-3.5 multidatabase feature, and has no meaning or use apart from its multidatabase role. That's better explained in the ZConfig <description> section for the key than in the name of the key, though.

If Jim doesn't object soon, I'll proceed with adding a database-name key to ZODB's config.

On Fri, 2005-10-21 at 11:13 -0400, Jim Fulton wrote:

...

Not really: a DB's database_name was introduced specifically for the new-in-ZODB-3.5 multidatabase feature, and has no meaning or use apart from its multidatabase role. That's better explained in the ZConfig <description> section for the key than in the name of the key, though.

If Jim doesn't object soon, I'll proceed with adding a database-name key to ZODB's config.

Note that I don't have a strong opinion about this either way but I will note that at least Zope 2's subclass of the "zodb" config handler will need to continue to be willing to use the section title as the database name for backwards compatibility reasons, as people who have older Zopes will want to use their older config files (which have zodb_db sections that have section titles, and no "database-name" key) with new Zope releases.

- C

[Chris McDonough]

There is a wrinkle about performing this merge that eluded my memory until now.

To support multidatabases within Zope, it was reasonable to change ZODB.config.ZODBDatabase to support the heretofore likely-unused-by-real-world-code "databases" and "database_name" options that may now be passed into ZODB.DB's constructor:

http://svn.zope.org/ZODB/branches/blob-merge-branch/src/ZODB/config.py?rev=3...

The current blob-merge-branch code depends on this change being available in the ZODB revision it uses. ...

Chris, here's the current state:

- The following is on current ZODB trunk, and on the new tag

svn://svn.zope.org/repos/main/ZODB/tags/3.6.0b1

I suggest changing the zodb-blobs-branch to use that tag now (and merging that branch to Zope trunk when it's all happy again).

- I added an optional new "database_name" key to <zodb> config. I understand that you may not want to use it in Zope 2.9, taking the database name from Zope's <zodb_db> section instead. That's fine. I expect that whatever name (however decided) should be used can be poked into config.database_name before calling ZODBDatabase.open(). Should check that the section name and database_name key aren't both specified? Probably, ya.

- ZODBDatabase.open() has an optional new databases= argument, so that part's still exactly the same way you did it (except that I didn't add that argument to BaseConfig.open() too -- I don't think it belongs there, as BaseConfig is also a base class for classes whose open() overrides don't support a databases= argument; the ZODBDatabase subclass _extends_ BaseConfig's open() in this respect).

Is there more I can do to help this along (or, perhaps, delay it more ;-))? If so, just ask.

Thanks for this!

Looks like that test failure is incidental and not symptomatic of changes made to ZODB. I think Tres may have said that it can be fixed by merging in a fix from the Five HEAD, but I don't know this for fact first-hand.

It's encouraging that most of the tests pass but there are a paucity of tests that specifically test Zope 2 multidatabase-based mount points. There are a few convincing-looking decoys in Products.ZODBMountPoint.tests but I think I'll need to create a few more to get the warm and fuzzies before doing the merge.

I have this on my plate for Wednesday evening.

- C

On Oct 24, 2005, at 2:45 PM, Tim Peters wrote:

Chris, FYI, I stitched ZODB 3.6.0b1 into zodb-blobs-branch, and changed ZopeDatabase.createDB() to plug database_name into config instead of passing it to ZODBDatabase.open(). The checkin msg summarizes test results; since I haven't work on this branch before, I'm not sure what was expected here (I didn't get a clean test run before stitching 3.6.0b1 in either):

<http://mail.zope.org/pipermail/zope-checkins/2005-October/ 

029904.html>

How would you like to proceed?

I lied. Due to completely preventable circumstances, this merge won't be done tonight; instead, it will be done tomorrow evening.

- C

On Mon, 2005-10-24 at 16:41 -0400, Tim Peters wrote:

[Chris McDonough]

...

Thanks for this!

Not required, so long as I get to thank you for finishing it ;-)

...

Looks like that test failure is incidental and not symptomatic of changes made to ZODB. I think Tres may have said that it can be fixed by merging in a fix from the Five HEAD, but I don't know this for fact first-hand.

I'm sure that failure will go away by itself when you're working on the trunk instead of the branch. What I'd do now:

• Check out Zope trunk.

• Merge the branch into your trunk sandbox, and forget the branch.

• Fix merge conflicts. I got one, in datatypes.py, and I didn't know immediately what to do about it so stopped there. You'll have better luck ;-). Note that, under SVN, after you fix a conflict, you have to do "svn resolved path/to/conflicted/file"; that's a gimmick to make sure you don't forget about conflicts.

• "svn up" to make sure you've got all the externals the merged files point at.

• "svn up" from time to time thereafter, to suck in other trunk changes as they get made.

• Check it in when it's stable.

• If it takes longer than expected, make a _new_ branch _from_ your merged-into-trunk local trunk sandbox. (That's easy: make a branch directory, "svn switch" to it from your local merged trunk sandbox, and "svn commit" -- all done).

...

It's encouraging that most of the tests pass but there are a paucity of tests that specifically test Zope 2 multidatabase-based mount points. There are a few convincing-looking decoys in Products.ZODBMountPoint.tests but I think I'll need to create a few more to get the warm and fuzzies before doing the merge.

As above, you can do a _local_ merge right away. This would save you from other decoys (like the DeprecationWarnings that would no longer exist if you were using the trunk instead of the brach, and the failing-on-branch-but-not-trunk Five test).

I recall that, historically, the Zope tests never failed when Zope mounting was in fact broken, so a fat +1 to beefing test coverage there.

...

I have this on my plate for Wednesday evening.

Understood; there really isn't any good TV on Wednesdays anymore ;-)

[Chris McDonough]

FWIW, I know a couple of people are depending on this, so here's an update.

I am working on merging multidatabase support, but I'm having some merge/update troubles (if you're interested in the symptoms, see http://www.plope.com/Members/chrism/heres_to_cvs). I suspect I'll work it out, but I've got my nose in Subversion documentation at the moment.

Jim redid the way Zope trunk stitches in Zope3 since you last looked at this, and that can create some mechanical problems (of the kinds you're seeing, in fact). The svn docs probably won't help. Suggestion (which is repetition of what I suggested before this happened, but we'll gracefully let that pass ;-)):

Check out a fresh, new copy of Zope trunk.

Merge your branch into that.

End of story. Unless you feel you need to make another branch. In that case, still do the two steps above first. Then create a new branch from Zope trunk, "svn switch" your merged sandbox to that new branch, then "svn checkin".

This way you shouldn't have any of the problems you've been seeing. If you do, double your money back ;-)

Jim redid the way Zope trunk stitches in Zope3 since you last looked at this, and that can create some mechanical problems (of the kinds you're seeing, in fact). The svn docs probably won't help. Suggestion (which is repetition of what I suggested before this happened, but we'll gracefully let that pass ;-)):

Check out a fresh, new copy of Zope trunk.

Merge your branch into that.

Thank you!

Eagds. I *thought* I had done just that. I had even printed out your previous handholding email about this before starting. But no. I did not. Instead, to my horror, I realized had *copied* a trunk working copy (of an unknown vintage, although up-to-date according to "svn status" after an svn up) and then I'd merged the branch into that.

So about a minute ago, I followed your instructions literally, figuring that I'd be able to sheepishly move on afterwards, but unfortunately it comes out the same. Literally, here are the commands:

$ svn co svn+ssh://svn.zope.org/repos/main/Zope/trunk $ cd trunk $ svn merge -r 38624:HEAD svn+ssh://svn.zope.org/repos/main/Zope/ branches/zodb-blobs-branch $ svn up

... which comes out with ....

....

Fetching external item into 'lib/python/zope/thread' External at revision 39684.

Fetching external item into 'doc/ZEO' A doc/ZEO/cache.txt A doc/ZEO/ZopeREADME.txt A doc/ZEO/README.txt A doc/ZEO/trace.txt A doc/ZEO/howto.txt Updated external to revision 39684.

Fetching external item into 'lib/python/zope' svn: Working copy 'lib/python/zope' locked svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details)

I am reasonably confident that drinking just one more Yuengling will solve this problem in one way or another.

- C

On Oct 28, 2005, at 9:19 AM, Jim Fulton wrote:

svn:externals suck. A lot. As Tim suggested, you could throw away this check out and start over. A simpler thing you could do is to remove the zope directory and do an svn up.

That sounds reasonable, but I've done both of those things and no joy yet. I've even switched machines in a hail-mary "maybe it's a local client bug" idea.

Since we switched to using externals, we see lots of things like this. You just learn to delete the directories in question.

I wish that worked for me. I'm going to try to offer the directory a cold glass of milk laced with arsenic next. We're getting so cozy with each other, I think it might accept.

- C

On Fri, 2005-10-28 at 10:12 -0400, Tim Peters wrote:

Ah, it's the properties on lib/python that are screwing us here! Chris, throw

svn revert lib/python

into the mix too. That got me unstuck. The problem is that both Jim and I (at least) changed the set of externals listed in lib/python, and SVN absolutely sucks at merging property changes. The "revert" above gets you back to the externals Zope trunk actually uses today. That's vital because Zope trunk stitches in "zope" in an entirely different way now. Reverting also stitches in a version of ZODB we don't want to use, but after reverting you can do

svn propedit svn:externals lib/python

to put back "the right" version of ZODB for your branch (s/3.4.2/3.6.0b1/g).

Woo hoo! As always, you are my hero, Tim. That worked great. And it as a bonus Zope even *starts* after changing the properties and running svn up.

- C

[Chris McDonough]

This merge has been done.

Woo hoo! Thank you, Chris!

I since stitched in ZODB 3.6.0b2, which is the most recent 3.6 (internal) release. That didn't seem to create any new problems.

Since "zopectl test <ProductName>" no longer appears to do the right thing

Sorry, never used it, don't know anything about it.

and I can't seem to get test.py to run anything except the entire test suite,

Do

test.py -h

for help. The -t option will probably help you. For example,

""" $ \python24\python.exe test.py -vv -t test_pop Running tests at level 1 C:\python24\lib\whrandom.py:38: DeprecationWarning: the whrandom module is deprecated; please use the random module DeprecationWarning) Running unit tests: Running: test_pop_default (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_raises (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_simple (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_validates (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_raises (AccessControl.tests.testZopeGuards.TestListGuards) test_pop_simple (AccessControl.tests.testZopeGuards.TestListGuards) test_pop_validates (AccessControl.tests.testZopeGuards.TestListGuards) Ran 7 tests with 0 failures and 0 errors in 0.000 seconds.

$ \python24\python.exe test.py -vv -t pop_val Running tests at level 1 C:\python24\lib\whrandom.py:38: DeprecationWarning: the whrandom module is deprecated; please use the random module DeprecationWarning) Running unit tests: Running: test_pop_validates (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_validates (AccessControl.tests.testZopeGuards.TestListGuards) Ran 2 tests with 0 failures and 0 errors in 0.000 seconds. """

I didn't create any new tests because I wouldn't have had the time to create tests and run them iteratively.

Sorry to ruin your weekend, then ;-)

That said, all existing tests pass and I did do some interactive stress-testing of the sessioning mount point, both of which made me feel comfortable enough to go ahead and do the merge.

The tests appear to be in exactly as good shape on Windows now as they were before (there are test failures on Windows; opened a Collector issue about them Thursday), so you have a plausible defense in court. I won't sue you if you won't sue me ;-)

Chris McDonough wrote:

This merge has been done.

Since "zopectl test <ProductName>" no longer appears to do the right thing

Could you be more specific?

cd ~/z2/2/ bin/zopectl test CMFCore Running tests via: /usr/local/bin/python2.3 /home/jim/z2/2/bin/test.py -v --config-file /home/jim/z2/2/etc/zope.conf CMFCore Parsing /home/jim/z2/2/etc/zope.conf Running tests at level 1 Test-module import failures:

Module: Products.CMFCore.tests.test_all

TypeError: Invalid test_suite, None, in Products.CMFCore.tests.test_all

Running unit tests: Running: .................................................. .................................................. .................................................. .................................................. ........................./home/jim/z2/2/Products/CMFCore/tests/test_PortalContent.py:78: DeprecationWarning: This will be removed in ZODB 3.7: subtransactions are deprecated; use transaction.savepoint() instead of transaction.commit(1) transaction.commit(1) # Make sure we have _p_jars .................../home/jim/z2/2/Products/CMFCore/tests/test_PortalFolder.py:538: DeprecationWarning: This will be removed in ZODB 3.7: subtransactions are deprecated; use transaction.savepoint() instead of transaction.commit(1) transaction.commit(1) ./home/jim/z2/2/Products/CMFCore/tests/test_PortalFolder.py:469: DeprecationWarning: This will be removed in ZODB 3.7: subtransactions are deprecated; use transaction.savepoint() instead of transaction.commit(1) transaction.commit(1) /home/jim/z2/2/Products/CMFCore/tests/test_PortalFolder.py:488: DeprecationWarning: This will be removed in ZODB 3.7: subtransactions are deprecated; use transaction.savepoint() instead of transaction.commit(1) transaction.commit(1) ..... ........./home/jim/z2/2/Products/CMFCore/tests/test_PortalFolder.py:893: DeprecationWarning: This will be removed in ZODB 3.7: subtransactions are deprecated; use transaction.savepoint() instead of transaction.commit(1) transaction.commit(1) ....../home/jim/z2/2/Products/CMFCore/tests/test_PortalFolder.py:1119: DeprecationWarning: This will be removed in ZODB 3.7: subtransactions are deprecated; use transaction.savepoint() instead of transaction.commit(1) transaction.commit(1) # get a _p_jar for 'sub' ................................... .................................................. .......... Ran 360 tests with 0 failures and 0 errors in 10.558 seconds.

Test-modules with import problems: Products.CMFCore.tests.test_all

Note that I may have updated the testrunner since you tries this. You might try again.

and I can't seem to get test.py to run anything except the entire test suite,

That's odd. What did you try?

As others have suggested, you should use -h to get help on the many options. I've tried to make the new test runner backward compatible with the old, but some old options are no longer supported.

Jim

...

[Chris McDonough]

Thanks for the offer! I won't be able to visit ZC world HQ tomorrow, though unless you'd be there and willing to start around 10pm.

Alas, they're still under the delusion that 10 _am_ is "late" here, so while I agree 10pm is saner on all counts, I'll be gone before then.

"Other duties" is my official excuse but I'm also horrified by the idea that I'd be expected to wear pants if I came over there. That's just uncivilized. :-)

There was such a backlash against the "pants required" policy that it's OK to wear diapers instead now. Progress, anyway.

...

[Tim]

...

Check. Question: does zodb-blobs-branch contain anything you _don't_ want to see on Zope trunk now? You didn't mention anything like that here.

No (save for inappropriate svn:externals to ZODB and ZEO).

In that case, and since you say later there's no reason to keep this branch after the merge is done, I suggest merging directly from zodb-blobs-branch to Zope trunk. Unless I'm missing something, there really doesn't seem to be a point to creating another intermediate branch first.

...

Yes. The zodb-blobs-branch can just die after this merge if there's a way to get delete branches entirely.

Yes and no ;-) "A branch" or "a tag" in SVN is just another named directory, with non-mandatory conventions for choosing its path name. It can be deleted, like any other directory. That doesn't get rid of it entirely, just as no directory can be gotten rid of entirely: you can always revert the checkin in which it was deleted, and then it will magically reappear. Unlike as under CVS, though, deleted branches don't keep punching you in the face if you don't go out of your way to find them. For example, these are all the visible ZODB branches today:

$ svn list svn://svn.zope.org/repos/main/ZODB/branches 3.3/ 3.4/ 3.5/ blob-merge-branch/ ctheune-blobsupport/

Note that none of the other branches we created at the ZODB sprint, or most of the branches created since then, still show up (I deleted them after merging to then-current ZODB trunk). SVN is very nice this way!

If there is to be a long-lived branch, it will be the "blob-merge-branch" of ZODB.

Afraid so, yes. Is it time to delete the ctheune-blobsupport branch?

Given that Zope 2.9 is not going to ship with blob support due to feature freeze, I think this means that we have until May to allow the blob-merge-branch to get utterly out of sync with the ZODB trunk. We can then easily wait until, say, the last week in April to worry about issues caused by that desynchronization. The work necessary to remerge should provide just the appropriate amount of delay to allow blobs to miss the next major Zope release. ;-)

Excellent! I'm glad you're thinking hard about this -- it gets hard delaying release after release all by myself ;-)