[Zope] Re: Stepper

[Chris Withers]

Hi Gary,

Gary Poster wrote:
> For our use case, the (quite frequent) cron kick could get to be  very, 
> very painful with long running tasks that may overlap (think  *lots* of 
> ConflictErrors).  

Ah well, if they're doing the same thing, just kicked off frequently, 
and you're worried about one run not finishing before another, I'd just 
put in some locking in Stepper that wouldn't run a chain/step if it was 
already running elsewhere, optionally logging this as an error...

> the pain carefully.  I'm reasonably confident that  Stepper would need 
> to become a long running process with a queue in  order to get there.  

I don't think it needs to be a long running process, and I think the 
queue could live elsewhere...

> Stepper is currently arguably better if you only have tasks like the  
> ones you list, as long as you only start Stepper at a slow enough  
> frequency that you have no chance for overlap (once a day sounds  
> reasonable, though who knows).  If you might overlap, zasync is safer  ATM.

The "locking prevents overlap" thing has worked really well for me in 
the past...

> zasync ended up being *very* careful, because we discovered we needed  
> to be.  Maybe a simpler, second gen design would need less care.

I tend to prefer logging errors and failing as much as possible. 
Anything else means Stepper wouldhave to do guesswork and that would be 
bad...

> On the other hand, if you are interested in zasync's through-the- 
> browser use case, maybe Stepper can grow a next-gen through-the-web  API 
> also. 

Nope, Stepper will never grow that. It's use case is much closer to that 
of "zopectl run" on steroids, but I think it could work well with a more 
light-weight zasync...

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk