[Zope] long running methods

[Maik Derstappen, Derstappen IT]

Am 20.05.2010 20:04, schrieb robert rottermann:
> Am 20.05.2010 19:34, schrieb Maik Derstappen, Derstappen IT:
>    
>> Am 20.05.2010 16:05, schrieb Jürgen Herrmann:
>>      
>>> On Thu, 20 May 2010 15:59:58 +0200, Patrick Gerken
>>> <do3ccqrv at googlemail.com>   wrote:
>>>
>>>        
>>>> On Thu, May 20, 2010 at 14:18, Jürgen Herrmann
>>>> <Juergen.Herrmann at xlhost.de>wrote:
>>>>
>>>>
>>>>          
>>>>> hi there!
>>>>>
>>>>> as our zope2 zodb grows we experience more and more conflicts
>>>>> with long running transactions. so i'm going to rewrite these
>>>>> methods to commit transactions in batches - here my attempt
>>>>> to tackle this problem. the idea is to get a list of objects
>>>>> to be modified in this transaction, break it down to batches
>>>>> and commit after each batch was processed. to avoid zope's
>>>>> retry-on-conflict machinery and still have a chance to retry
>>>>> per batch transaction commits, the iteration over one batch
>>>>> is guarded by a try:except ConflictError: and a retry logic.
>>>>>
>>>>>
>>>>>            
>>>> Zope already has its own retry mechanism, maybe you might want to
>>>> call your long running method multiple times, each time working on the
>>>> next batch.
>>>>
>>>>
>>>>          
>>> in fact zope's retry machinery is kicking in too often which is a
>>> bad thing if the retried transaction runs for say 30min or so.
>>>
>>> having to gather the list of objects outside of zope, doing the
>>> batching there and calling into zope multiple times - i wanted
>>> to avoid that.
>>>
>>> i'd like to have all the logic contained in one zope method
>>> for the sake of simplicity on the caller's side (most of these
>>> are kicked off by cron jobs).
>>>
>>> jürgen
>>>
>>>        
>> take a look at http://pypi.python.org/pypi/affinitic.zamqp maybe it is a
>> solution for you.
>> You can deligate your jobs out of zope and get it back after some time.
>> zamqp then fire an event and you subscribe this event to handle thie
>> results.
>>
>> with kind regards
>> Maik Derstappen
>>
>>      
> maik,
> this looks interesting.
> just out of curiosity: how would one alert the user that a long running job (say
> after some 10 minutes) has finished.
> does the client have to poll?
> or is there some "magic" push functionality?
>
> robert
>    
hi robert,

zamqp has an process that continuously receive new message from his 
message queue and call a callback method which fire the event, that you 
can subscribe.

see zamqp and carrot docs for more info ;)

regards maik

-- 
========================================================================

Derstappen    I T  Consulting   Tel:         +49 ( 341 )   600  13  0 31
Zope/E-Mail/Backup/Monitoring   Mobil:       +49 ( 178 )   861  2    833
M a i k   D e r s t a p p e n   Fax:         +49 ( 180 ) 5 021 121 90 56
H e r l o ß s o h n s t r  12   Email:  maik.derstappen at derstappen-it.de
0 4 1 5 5       L e i p z i g   Internet:    http://www.derstappen-it.de

========================================================================