[Zope] SQLSession vs FSSession for ultra-high scalability and speed.

[J. Atwood]

Hi Chas,

Let me give it a try (or just dump all of what I know).

First of all it looks like you are going to want ZEO. If you don't 
know ZEO, get to know ZEO, it fits perfectly with what you want for 
the frontline servers.

As for the SQLSessions. Since there is a lot of writing in 
SQLSessions I would suggest that you stick to the RDBMS model, Zope 
can handle it, but SQL can handle it better and if you are already 
planning on putting a DB behind the Zope installation it will really 
not take that much more (if anything) to add session information. You 
might, however, depending on time, want to invest  a little in 
playing around with your own SQL code to handle session data. There 
are factors like Zope's ability to cache that you will want to take 
full advantage of.

As for the database, MySQL is hands down the fastest, but certainly 
not the most robust in terms of features. Oracle is great if you have 
$10,000 and a full time DBA but PostgreSQL 7.0 is really your best 
bet. You might also want to take the money you were going to spend on 
the DB2 and put it right back into DB1 and house everything there. 
Just get a beefy machine, RAID it up for redundancy and call it a day.

Take a look at some of the benchmarks I ran for an idea of what you 
can do with a plain ole RH 6.1 box and PostgreSQL.

http://www.zope.org/Members/BwanaZulia/zope_benchmarks/benchmarks.html

The other idea that I have been playing around with in terms of 
redundancy is to build something into the Zope code for the ZSQL 
Methods that can go into a "cached" mode if your database dies. I 
have only begun to play with this, but in theory since, Zope can 
cache all your common data that comes from SQL you could, in effect, 
bring down the SQL database and have Zope continue to work with the 
cached data. I think all that it would involve (and I am sure that I 
am simplifying the hell out of this) is to have Zope know that the 
SQL database is no longer there (look for an error) and just turn the 
"Maximum time (seconds) to cache results" to unlimited until the 
database was restored.

Has anyone else thought about this or played with this idea?

Ok.. that is enough...

Cheers,
J



>Sorry if this has been asked before, but can anybody
>advise on FSSession vs SQLSession for:
>
>a) Speed.
>b) Scalability.
>
>Planned architecture employs multiple Zope/Apache
>frontline servers:
>
>+-----+
>| Z/A |---+
>+-----+   |
>           |
>+-----+   |           
>| Z/A |---+      +----- DB1
>+-----+   |      |
>           |------|
>+-----+   |      |
>| Z/A |---+      +----- DB2 (optional)
>+-----+   |
>           |
>           |
>       Etc, etc.
>
>Z/A = 'front line' webservers running Zope/Apache.
>       Contains application/business logic.
>       We are NOT using the ZODB for data storage.
>
>DB1 = Data storage. eg. Oracle/Postgres
>
>DB2 = potentially just for managing the sessions, if
>       we use SQLSession. Would MySQL be best ?
>
>
>Just how scalable can it really get ? We need it to at least
>be able to scale to 12 million sessions per day (and substant-
>-ially more writes to DB1 per day), even if we will be starting
>out much more modestly than that.  Realistically, should I be
>looking at something like ATG Dynamo instead ? 
>
>Thank you very much,
>
>chas
>
>_______________________________________________
>Zope maillist  -  Zope@zope.org
>http://lists.zope.org/mailman/listinfo/zope
>**   No cross posts or HTML encoding!  **
>(Related lists -
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope-dev )