[Grok-dev] Re: ANN: New Grok Powered Website

[Kevin Smith]

Hi Martijn,


Martijn Faassen wrote:
> Hey,
>
> Thanks for letting us know about this website! Once every while I 
> wondered how you were doing as you were rather quiet on the list, so I 
> am very happy to see that the site is up and running.
>
> Kevin Smith wrote:
> [snip]
>> Biggest hurdles: lack of export/import, moving large data sets,
>
> I'd be nice if you could expand on this: what kind of import/export 
> system would you need? (what would be sufficient, what would be 
> ideal?) How did moving large data sets cause problems?
>
I shouldn't have included moving large data sets as that had nothing to 
do with Grok/Zope3. Large data sets are just unwieldy.

As far as import/export is concerned, I was very happy with Zope2's 
.zexp files. AFAICT the only thing that prevents the ZODB built-in 
import/export mechanism from working is that it pickles __parent__ which 
means exporting *anything* exports the *entire* database. 

I actually had basic pickling working for awhile but didn't have the 
time/know-how to isolate the issue to see if an upgrade to Grok/Zope3 or 
programming on my part broke it.

I did see stumble across your "secret"  work in Zope SVN on an export 
alternative, but from what I saw took more manual configuration than I 
had time to implement.

>> lack of docs/examples (at the time), 
>
> Can still be much improved, but we've been moving forward on this indeed.
>
>> lack of layers and viewlets,
>
> Thank you here for the contributions! Layers have landed, and viewlets 
> will do so soon enough. You should thank Tim for pushing me at the 
> Snow Sprint into looking into them - I'm quite pleased with the 
> progress we made there.

Thank you Tim!
>
>> Catalog scaling limitations
>
> This is interesting to find out more about. Any ideas as to how to 
> approach a solution? A common way that people have approached this 
> problem is to use a relational database (or custom index software such 
> as lucene) to back up the catalog index. I think it wouldn't be very 
> hard to come up with a set of indexes that are RDB backed. Harder, but 
> possible would be to make hurry.query aware of them so that it create 
> smart SQL queries.
My solution was to create a new Catalog for each year's worth of 
content. For my use case this works. I didn't want to add the additional 
overhead of real RDB interactions and lucene which may make a more 
generalized solution.

>
>> lack of respect to how long it takes to build a custom CMS from 
>> ground up
>
> :)
>
>> zc.buildout
>
> Already discussed elsewhere in the thread; I'm glad you are happy now 
> we have our dependencies under control.
>
>> Biggest helpers: GROK SMASH ZCML, grok.View, hurry.query, groklets, a 
>> renewed respect for ZPT's especially macros, Martijn's sense of 
>> humor, Philipp's book, emacs, screen, open source in general, 
>> zc.buildout
>
> I'm glad you think I got a sense of humor. :)
>
> [moving this bit from the bottom so I can comment on it in a better 
> context]
> > Grok 0.9 (heavily expanded to use layers, viewlets, groklets,
> > additional
> > field indexes and add 'context' to hurry.query)
>
> Could you elaborate on the additional field indexes you created, and 
> you mean by adding context to hurry.query?

Unless they've been added since 0.9, I've cherry-picked a few of the 
field indexes from zc.catalog on an as needed basis.
Content is divided by year. So each year is  a site with it's own 
/++etc++site and it's own Catalog. Currently hurry.query doesn't allow 
for context although getUtility does. Here's how I tweaked hurry.query

class IndexTerm(Term):
    def __init__(self, (catalog_name, index_name), context=None):

        self.catalog_name = catalog_name
        self.index_name = index_name
        self.context = context

    def getIndex(self):

        catalog = zapi.getUtility(ICatalog, self.catalog_name, 
context=self.context) #using context
        index = catalog[self.index_name]
        return index

class Text(IndexTerm):
    def __init__(self, index_id, text, context=None):
        super(Text, self).__init__(index_id, context=context)
        self.text = text

    def getIndex(self):
        index = super(Text, self).getIndex()
        assert ITextIndex.providedBy(index)
        return index

    def apply(self):
        index = self.getIndex()
        return index.apply(self.text)


>
> > The site initially made heavy use of viewlets which were then
> > simplified to groklets.
>
> It'd be valuable to see your feedback on the viewlets branch. I'd like 
> to hear more about how the groklet simplification helped you. We 
> decided not to do away with the viewletmanager concept yet as it could 
> be made to be quite small with Grok. I noticed that your viewlets 
> branch had some code that seemed to break some features of viewlets (I 
> forget the details, I think possibly the ability to associate them 
> with particular views, but I may be wrong).
For the Zope3 developer, viewlets are a valuable addition to Grok. 
Groklets aren't ready for general consumption yet, but it did cut down 
my 'viewlet' code by at least a third.
>
> I hope that you won't be stuck on 0.9 forever and that you will be 
> able to move to Grok 1.0+ so we share a common base for these features.
yes, hopefully before Grok 2.0 anyway :)
>
>> The site's not finished yet, and our business requirements are 
>> constantly in flux, but thankfully due to Grok's roots in Zope3, it's 
>> always being evolved to meet our needs.
>>
>> THANK YOU Grok community for your hard work and know-how in creating 
>> a dream web programming environment.
>>
>> Go GROK!
>
> Thank you very much for all the contributions you've already made for 
> Grok, and this feedback - it's valuable feedback and also quite 
> pleasant to see Grok worked out for you!
Thanks you, and hopefully I'll be able to open time to contribute 
properly in the future ( I don't even remember implementing the viewlets 
branch!)

>
> Regards,
>
> Martijn
>
>
> _______________________________________________
> Grok-dev mailing list
> Grok-dev at zope.org
> http://mail.zope.org/mailman/listinfo/grok-dev
>