Give it a rest + answers. (Re: [Zope] Re: Zope + Apache on Quad Debian machine)

[Dario Lopez-Kästen]

Jeff Donsbach said the following on 2006-03-21 17:26:

> Dario,
>     Do you have any kind of comparison numbers of using CPU affinity
> vs not for your particular case? Also, are you using ZEO or not?  It's
> not that I don't believe you when you say it matters a lot for you. I
> do believe you. Like Tino, I'm just generally interested in how much
> it matters in measurable terms. I can imagine there are a number of
> factors determining how much it matters, like Zope app/workload as
> well as the underlying hardware architecture (how big of a penalty is
> it to synchronize cache pages between CPUs) and the OS CPU scheduler
> as Tino mentioned.

To be honest, we have never had the inclination to do much of research 
in this area. Our situation has mostly been such that we experience 
horrible delays in zope-responsiveness in testing that vanish as soon as 
we use taskset. This is on both Solaris (with the equivalent of taskset 
there) and Linux .

Since then (and for other reasons), we have migrated all of the central 
servers we manage from Solaris to Linux, so I cannot give any more input 
about Solaris.

Yes, we use ZEO almost exclusively, it is realtively easy to setup even 
fpr development and we don't deploy without ZEO anymore. It makes a fair 
bit of improvement as well.

We have an app (the one that tooks us on the road to zope) that for 
various reasons has not updated properly since 2002 when we first had 
our multicpu experiences.

This particular app receives quite a bit of load, and since it is built 
entirely thru the web with by now age-old DTML and 
ye-olde-zope-techniques, it is not the speediest in the world. Add to 
that the fact that we use DCOracle2(*) to do Oracle queries, and we 
sometimes expericence hangups on the ZEO clients.

For this app, we have "successfully" delayed it's demise and avoided 
total chaos by adding more ZEO clients (at the moment, these clients 
runt on two machines, four processes on one and two processes on the 
other). Well, we also have scripts that restart the nodes when the leak 
memory too much, and so on. BUT: the speed of the app has increased with 
each node we add.

I am sorry that I cannot give more numbers - I hear from the traffic on 
the list that there are other factors involved nowadays that may in some 
way obsolete the need to bind to a particular CPU, however I do not 
understand how this can have an impact on the GIL.

Let me be the first to admit my total lack of knowledge of kernel task 
schedulers, but generally speaking, unless the scheduler makes sure that 
a threaded python process never ever gets distributed over two 
processors simultaniously,  then the GIL *will* be an issue.

In any event, I'd love to be proved wrong about the need for taskset 
(especially if someone comes up with measurable data to that effect) 
because it means that my sysadmins can simplify their setup for managing 
Zope (making Zope even more acceptable :-).

Cheers,

/dario

(*) We have not been able to use the versions of DCOracle2 that ChrisW 
has worked on, because we expericend nother set of problems with them 
and we never had the opportinuty to really spend time chasing those 
bugs. I believe ChrisW's DCO2 does solve some of the issues that the 
original DCO2 has

Please note that the problems we had with it, may very well becasue of 
our particular setup (Oracle 8, bad code in our app, even worse sql, 
etc). I have heard that for other people Chris's DCOracle2 versions work 
better than the non-modified DCO2, so YMMV.

-- 
-- -------------------------------------------------------------------
Dario Lopez-Kästen, IT Systems & Services Chalmers University of Tech.
Lyrics applied to programming & application design:
"emancipate yourself from mental slavery" - redemption song, b. marley