[Zodb-checkins] CVS: StandaloneZODB/bsddb3Storage - README:1.9

[Barry Warsaw]

Update of /cvs-repository/StandaloneZODB/bsddb3Storage
In directory cvs.zope.org:/tmp/cvs-serv32602

Modified Files:
	README 
Log Message:
Major update in preparation for ZODB 3 beta 1.


=== StandaloneZODB/bsddb3Storage/README 1.8 => 1.9 === (414/514 lines abridged)
--- StandaloneZODB/bsddb3Storage/README:1.8	Tue Feb 12 17:33:08 2002
+++ StandaloneZODB/bsddb3Storage/README	Fri Aug 23 13:02:28 2002
@@ -1,169 +1,133 @@
-Berkeley (bsddb3) Storage for ZODB/Zope
+Berkeley Storage for ZODB
 
 Please see the LICENSE.txt file for terms and conditions.
 
 Introduction
 
-    This package contains implementations for Zope/ZODB storages based
-    on Sleepycat Software's BerkeleyDB and the PyBSDDB3 extension
-    module.  These storages save ZODB data to a bunch of BerkeleyDB
-    tables, relying on Berkeley's transaction machinery to provide
-    reliability and recoverability.
-
-    WARNING: We do not consider Berkeley storage to be production
-    quality yet.  The primary outstanding issue is poor performance
-    due to the underly Berkeley database's inability to efficiently
-    handle large `blobs'.  ZODB is designed to store the pickles of
-    persistent objects, and these pickles can sometimes be quite
-    large; pickles on the order of 10's of KB or even several MB are
-    not unheard of.
-
-    We are currently working with Sleepycat to resolve this issue, but
-    as of yet have not arrived at a satisfactory solution.  Be aware
-    that if your persistent objects have pickles over 4K in size
-    (which is likely), you will experience suboptimal performance.
+    This package contains implementations for ZODB storages based on
+    Sleepycat Software's BerkeleyDB and the PyBSDDB3 extension module.
+    These storages save ZODB data to a bunch of BerkeleyDB tables,
+    relying on Berkeley's transaction machinery to provide reliability
+    and recoverability.
+
+    Note that the Berkeley based storages are not "set and forget".
+    The underlying Berkeley database technology requires maintenance,
+    careful system resource planning, and tuning for performance.
+    You should have a good working familiarity with Berkeley DB in
+    general before trying to use these storages in a production
+    environment.  It's a good idea to read Sleepycat's own
+    documentation, available at
+
+	http://www.sleepycat.com
+
+    See also our operating notes below.
 
 
 Contents
 
-    Inside the bsddb3Storage package, there are three storage

[-=- -=- -=- 414 lines omitted -=- -=- -=-]

+    default page size for BerkeleyDB, while system dependent, is
+    typically 4K.  Any objects larger than about 4K must be broken up,
+    with pieces of the object stored on overflow pages.
+    Unfortunately, Python pickles can indeed be larger than 4K, and
+    we've witnessed some object pickles growing as large as 25MB.  We
+    have noticed a performance degradation when writing huge numbers
+    of overflow pages.
+
     One thing we can safely say is that the default Berkeley cache
-    size of 256KB is way too low to be useful.  Be careful setting
-    this too high though, as performance will degrade if you tell
-    Berkeley to consume more than the available resources.  For our
-    tests, on a 256MB machine, we've found that a 128MB cache appears
-    optimal.  YMM(and probably will!)V.
+    size of 256KB is probably too low to be useful.  Be careful
+    setting this too high though, as performance will degrade if you
+    tell Berkeley to consume more than the available resources.  For
+    our tests, on a 256MB machine, we've found that a 128MB cache
+    appears optimal.  YMM(and probably will!)V.
 
-    As you read this, it will be helpful to know that the
+    As you read these, it will be helpful to know that the
     bsddb3Storage databases all use BTree access method.
 
 
@@ -424,11 +400,19 @@
     Linux ext2-based systems.  The Berkeley storage pickle database
     (by default named "zodb_pickle"), which holds the bulk of the data
     for the Berkeley storages is particularly susceptible to large
-    growth.  If you notice that this file's size (or any other
-    Berkeley storage-related file) is nearing 2GB, you'll need to move
-    your BerkeleyDB environment to a filesystem which supports > 2GB
-    files.  If you enable largefile support for your filesystem, you
-    should be able to create much larger databases.
+    growth.
+
+    If you anticipate your database growing larger than 2GB, it's
+    worthwhile to make sure your system can support files larger than
+    2GB.  Start with your operating system and file system.  Most
+    modern Linux distributions have large file support.
+
+    Next, you need to make sure that your Python executable has large
+    file support (LFS) built in.  Python 2.2.1 is automatically
+    configured with LFS, but for Python 2.1.3 you will need to rebuild
+    your executable according to the instructions on this page:
+
+    http://www.python.org/doc/2.1.3/lib/posix-large-files.html
 
     IMPORTANT NOTE: If any of your BerkeleyDB files reaches the 2GB
     limit before you notice the failure situation, you will most