[Zope-dev] Handling of ECONNRESET socket error in asyncore.py

[Oliver Bleutgen]

Chris McDonough wrote:

> On Fri, 2002-08-23 at 17:59, Oliver Bleutgen wrote:
> 
>>Hmm, how should the browser come back? Where to? The TCP-connection is 
>>closed. You have no way to get data back to the browser. Press the stop 
>>button, tear the network out of the wall, the thread won't stop.
>>
> 
> When you come back, you'd come back to another thread, but presumably
> the other thread would either still be running or will have finished.


Yes, and where does it put his output to? There's no way to get it 
directly to the browser. Therefore, the typical situation is 
pdf-generation for later delivery (via email etc.) or whatnot, and that 
doesn't need to waste one of zope's worker threads, they seem too 
precious for me.


> 
>>Maybe you say that someone may want to start a long running job per 
>>browser request, like report creation or mass mailing, but who on his 
>>right mind would implement something that way?
>>
> 
> Not sure if I'm in my right mind, but I cause long-running requests a
> fair amount.  Packing a large databases, reindexing large catalogs,
> creating PDFs... sometimes you can't avoid it.


Ok, you're completely right, sometimes you can't avoid it. This is 
normaly no problem, since the user knows that he will have to wait. But 
it's telling that 2 of your examples are administrative tasks.
As for pdf-creation, I've seen a lot of people on the mailing list doing 
stuff like that in the background, if the creation takes too much time. 
I should have said "very long running tasks, which may get started 
relativly often". This kind of tasks where you don't need to or can't 
send something back to the browser.

 
>>Imagine you have one slow/hanging page, maybe representing data from an 
>>overloaded sql-database connected to zope. As it is now, sooner or later 
>>all of zope's worker threads will end up hanging there, making the site 
>>inaccessible, while the users who requested the offending pages have 
>>shut down their clients long ago. Yes, I know, there may some other 
>>timeouts kick in, but the principle still stands.
>>
> 
> Right, there is a real issue here.
> 
> Did you read my suggestion for a new Zope feature e.g.
> REQUEST.RESPONSE.isClientConnected?  It would let the programmer
> explicitly decide when to stop processing based on the connection status
> of the browser.  Adding this feature would make it possible for a
> programmer to influence execution in arbitrary ways based on client TCP
> connection status without impacting backwards compatibility.  For
> example with your "slow/hanging page", maybe each iteration through a
> loop that obtains data from the database would check the flag and stop
> processing if the user has gone away, e.g.:

> 
> l = []
> while database.next():
>     # pretend "slow" inner loop
>     l.append(database.getData())
>     if not context.REQUEST.RESPONSE.isClientConnected():
>         return l
> return l


I don't see backwards compatibility compromised. If the job of a method 
is to send its results to the browser, connection reset means it can't 
fullfil it's job. If you don't need to send something to the browser, 
you better should have offloaded the work to a special thread/process 
anyway. Maybe the administrative tasks you mentioned above could just 
ignore the connection reset instead of getting stopped, but show me 
someone who has pressed the stop button, reloaded or went to another 
page while a database pack was in progress, and I show you a very brave 
person.
It's just super counterintuitive that the work continues after I pressed 
stop.

>>Apache + mod_perl don't do that, btw, and I bet a lot of other servers 
>>don't do also.
>>
> 
> Are you sure?  As far as I know, under mod_perl, the 'page' continues to
> be executed after the user presses the stop button.
> 
> The "mod_perl developer's cookbook" also recommends something like I'm
> suggesting with 'isClientConnected': 
> http://webreference.com/programming/perl/cookbook/chap4/5.html as well.


I'm not sure why they talk about this, but it might have to do with the 
special example they give. I.e. checking if the client is connected 
before kicking of an expensive database query. I think their recipe is 
to handle the fact that the script also doesn't get stopped immediately 
when the connection is closed by the client, but that doesn't mean that 
it runs indefinately.

As for if I'm sure that apache+mod_perl works different than zope, I 
tested that myself and wrote it to the list:

http://mail.python.org/pipermail/zope/2001-October/102360.html

And here's another page I just googled:

"Handling the 'User pressed Stop button' case
Ok, When user decided to press the stop button, apache will detect that 
thru the SIG{'PIPE'} and will cease the script execution. [...]"

http://www.dlhoffman.com/publiclibrary/software/apache-perl-guide/obvious.html#Handling_the_User_pressed_Stop_

I don't know how old this text is, but I think we safely can assume that 
apache didn't get worse since then ;-).

cheers,
oliver