[Grok-dev] remove the download tarball functionality from grokproject?

[Martijn Faassen]

Hi there,

Jan-Wijbrand Kolman wrote:
> Like Vincent reported, grokproject will download the big tarball 
> ("Eggbasket") even if (most of) the required eggs are already 
> available in your egg cache.
> 
> It is also an extra step in the release process to actually create the 
> eggbasket and upload it.
> 
> How would people feel if I were to remove this eggbasket downloading 
> from grokproject?

I've been thinking about this.

The reason we have the giant tarball is primarily reliability. We can 
host the tarball ourselves on grok.zope.org. If for someone reason some 
random website that hosts something we depends on is down, people can 
still install Grok. That's useful especially when people are installing 
Grok for the first time.

It's also clear that the giant tarball is a liability though. It makes 
our installation procedure more complicated. If we force people to 
download it each time even though they don't need all of it, that's 
silly too.

I'll note that the first-time installation group is not us. That's why I 
think we should think of them explicitly.

Leonardo was experimenting with an installation method which is a giant 
tarball, but where grokproject itself is *also* in that tarball. I.e. 
it's more like the traditional way people install stuff - they download 
the tarball, unpack it, and follow installation instructions.

I'm +1 on that approach and replacing the eggbasket approach. It'll be 
more familiar for first-time users than what we have now, and it won't 
bother the existing users.

My primary concern is that any such method should end up with an 
installation that makes sense *beyond* first time installation of Grok. 
It should allow getting stuff from PyPI, it should install shared eggs 
somewhere, the buildout.cfg that it generates shouldn't make assumptions 
that don't work for everyone (the rule that if I check my generated 
buildout.cfg into SVN you can check it out and run buildout to get the 
same results), etc.

Alternatively, we could introduce some reliability by running our own 
mirrored package index on grok.zope.org. That just sounds too "special" 
and cumbersome for us to maintain though.

The design of the installer could be that it creates a directory 
structure with all the required tgzs and binary eggs somewhere, and it 
can install a bin/grokproject. When bin/grokject is installed, all the 
source tgzs should be installed in a shared eggs dir somewhere (possibly 
under that directory). It's quite possible eggbasket can actually do 
what we want; we just only run it once during an explicit installation 
check, instead of running in each time we are creating a Grok project. 
So, we'd not have any eggbasket stuff in grokproject-generated buildout.cfg.

Can we make progress on such an installer?

Regards,

Martijn