[Zope-dev] is threading in zope useful ?

[sathya]

Dario Lopez-Kästen wrote:
thanks much for the info.
turns out that the updated kernel rpms for RH8 and RH 9  2.4.20-28.8smp 
contain the patch for cpu affinity as well
at this point seems like a good idea  to deploy ZEO and bind each client 
to a processor and  test it out ,hopefully with a 4way smp we should see 
4X improvement in performance !!! (We hope )

one other point
[dario]
 >As I understand it. the GIL is not a problem *at all* unless you 
happen to run on smp machines. There are ofcourse >optimisations to when 
and where to relase >the GIL, but in general python takes care of that 
all by itself.
How about when  you  use  extension modules that python does not have 
apriori knowledge about (it only optimizes when it encounters known 
"threadsafe" api calls like the ones in clib or socket lib ). So, as I 
understand it if a python database module invokes a native library call 
python will not yeild the GIL. So if your zope app is making heavy use 
of an RDBMS  we would still see zope threads waiting on the GIL until 
the long running thread fetched DB results and performed  network IO to 
send back the response at which point the GIL  would be released.  Some 
GIL aware extesnion modules such as  the mxodbc DA from egenix   free 
the GIL for query API 
(http://www.egenix.com/mailman-archives/egenix-users/2003-February/000200.html).  


That said it is clear that using ZEO and CPU affinity guarantees the 
best  utilization of SMP for your zope app

Thanks tim ,dario and christian ....

regards
sathya

> sathya wrote:
>
>>
>> What about threads on linux ? will binding a process to a CPU also 
>> mean all threads spawned by the process are also bound to the same 
>> processor ? I am not aware of linux support for binding threads to a 
>> CPU so I guess it does not matter .
>
>
> For a full discussion of this and oter SMP issues, see this article on 
> Zope org.
>
> http://www.zope.org/Members/glpb/solaris/multiproc
>
> It is possible to do CPU-affinity on Linux, and nowadyas there are 
> even userspace tools that do this.
>
> http://www.hpl.hp.com/research/linux/kernel/o1-openmp.php
>
> http://www.tech9.net/rml/
>
> http://kerneltrap.org/node/view/335
>
>
> CPU.-affinity is avilable in RH 3.0 per default, and the same tools 
> are avialable as a set of patches for various kernel versions.
>
> *In general, on an SMP box, Python processes MUST be bound to any one 
> of the CPUs on a amchine* Not the same CPU, but each python process 
> has to stay on the same CPU all the time.
>
> As I understand it. the GIL is not a problem *at all* unless you 
> happen to run on smp machines. There are ofcourse optimisations to 
> when and where to relase the GIL, but in general python takes care of 
> that all by itself.
>
> ZEO won't save you from the GIL - ZEO will allow you to scale your 
> Zope beyond the constraints of 1 singel zope process for serving the 
> same content.
>
> ZEO implements storage of the data Zope serves in one centralised 
> location (the ZEO storage server) and allows several Zope processes 
> (acting as ZEO clients) to serve/update the same content. These Zope 
> processes can ofcourse be placed in a kluster, each ZEO client in a 
> different machine.
>
>
> /dario
>