[Zope] long running methods

robert rottermann robert at redcor.ch
Thu May 20 14:04:06 EDT 2010


Am 20.05.2010 19:34, schrieb Maik Derstappen, Derstappen IT:
> Am 20.05.2010 16:05, schrieb Jürgen Herrmann:
>> On Thu, 20 May 2010 15:59:58 +0200, Patrick Gerken
>> <do3ccqrv at googlemail.com>  wrote:
>>
>>> On Thu, May 20, 2010 at 14:18, Jürgen Herrmann
>>> <Juergen.Herrmann at xlhost.de>wrote:
>>>
>>>
>>>> hi there!
>>>>
>>>> as our zope2 zodb grows we experience more and more conflicts
>>>> with long running transactions. so i'm going to rewrite these
>>>> methods to commit transactions in batches - here my attempt
>>>> to tackle this problem. the idea is to get a list of objects
>>>> to be modified in this transaction, break it down to batches
>>>> and commit after each batch was processed. to avoid zope's
>>>> retry-on-conflict machinery and still have a chance to retry
>>>> per batch transaction commits, the iteration over one batch
>>>> is guarded by a try:except ConflictError: and a retry logic.
>>>>
>>>>
>>> Zope already has its own retry mechanism, maybe you might want to
>>> call your long running method multiple times, each time working on the
>>> next batch.
>>>
>>>
>> in fact zope's retry machinery is kicking in too often which is a
>> bad thing if the retried transaction runs for say 30min or so.
>>
>> having to gather the list of objects outside of zope, doing the
>> batching there and calling into zope multiple times - i wanted
>> to avoid that.
>>
>> i'd like to have all the logic contained in one zope method
>> for the sake of simplicity on the caller's side (most of these
>> are kicked off by cron jobs).
>>
>> jürgen
>>
> take a look at http://pypi.python.org/pypi/affinitic.zamqp maybe it is a 
> solution for you.
> You can deligate your jobs out of zope and get it back after some time.
> zamqp then fire an event and you subscribe this event to handle thie 
> results.
> 
> with kind regards
> Maik Derstappen
> 
maik,
this looks interesting.
just out of curiosity: how would one alert the user that a long running job (say
after some 10 minutes) has finished.
does the client have to poll?
or is there some "magic" push functionality?

robert


More information about the Zope mailing list