<html>
<body>
<p>This is a feature matrix and comparison comparison of the various ZODB storages.</p>
<p><b>status:</b> Currently it is incomplete. I (Toby Dickenson) have filled in the columns
for the storages I know well, and rows for the features that differentiate them.
Please update the other columns, add new columns, and particularly add new rows if any other
areas of comparison are important.</p>
<p>Apologies in advance for any errors.</p>
<style>
.comparison th {
text-align: left;
}
.comparison .title {
background-color: #eee;
padding-top: 6pt;
};
.comparison .filestorage {
background-color: #A8D8F0;
};
.comparison .dirstorage {
background-color: #EBD7B9;
};
.comparison .bdbstorage {
background-color: #A8AFF0;
};
.comparison .ape {
background-color: #8FC877;
};
</style>
<table class="comparison">
<tr>
<th class="filestorage">
FileStorage
</th>
<th class="dirstorage">
<a href="http://dirstorage.sourceforge.net">DirectoryStorage</a>
</th>
<th class="bdbstorage">
<a href="http://www.zope.org/Wikis/ZODB/BerkeleyStorage">BDBStorage</a>
</th>
<th class="ape">
<a href="http://hathaway.freezope.org/Software/Ape">Ape</a>
</th>
</tr>
<tr><th colspan=4 class=title>Maintainer</th></tr>
<tr>
<td class="filestorage">
Jeremy Hylton
</td>
<td class="dirstorage">
Toby Dickenson
</td>
<td class="bdbstorage">
Barry Warsaw
</td>
<td class="ape">
Shane Hathaway
</td>
</tr>
<tr><th colspan=4 class=title>History</th></tr>
<tr>
<td class="filestorage">
ZODB's first ever storage.
</td>
<td class="dirstorage">
Based on an idea prototyped in 2000, became stable in 2002.
</td>
<td class="bdbstorage">
First versions released in 2001. Added to standard ZODB version 3.2.
</td>
<td class="ape">
Currently in alpha release.
</td>
</tr>
<tr><th colspan=4 class=title>Installed Users</th></tr>
<tr>
<td class="filestorage">
All ZODB Users.
</td>
<td class="dirstorage">
10's
</td>
<td class="bdbstorage">
?
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Active Developers</th></tr>
<tr>
<td class="filestorage">
10's
</td>
<td class="dirstorage">
1
</td>
<td class="bdbstorage">
2?
</td>
<td class="ape">
1?
</td>
</tr>
<tr><th colspan=4 class=title>License</th></tr>
<tr>
<td class="filestorage">
ZPL
</td>
<td class="dirstorage">
LGPL
</td>
<td class="bdbstorage">
ZPL
</td>
<td class="ape">
ZPL
</td>
</tr>
<tr><th colspan=4 class=title>Supported Platforms</th></tr>
<tr>
<td class="filestorage">
Everywhere.
</td>
<td class="dirstorage">
Unix only. Only linux is well tested.
</td>
<td class="bdbstorage">
?
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>ZODB3</th></tr>
<tr>
<td class="filestorage">
yes
</td>
<td class="dirstorage">
yes
</td>
<td class="bdbstorage">
yes
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>ZODB4</th></tr>
<tr>
<td class="filestorage">
yes
</td>
<td class="dirstorage">
not yet
</td>
<td class="bdbstorage">
yes
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Implementation Overview</th></tr>
<tr>
<td class="filestorage">
Data.fs is a log file. New object revisions are appended. An in-memory index maps from oid
to seek position within this file.
</td>
<td class="dirstorage">
Uses ordinary files and directories to store revisions of ZODB objects; one file per revision per object, plus
one file per transaction.
</td>
<td class="bdbstorage">
Uses the Sleepycat edition of BerkeleyDB. Approximately 20 tables are used to store the storage structures.
Some tables are reference counted, with reference counts stored in other tables.
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Design Goals</th></tr>
<tr>
<td class="filestorage">
High performance through low overheads. Extreme portability.
</td>
<td class="dirstorage">
Simplicity of implementation. Fault tolerance. Manageability.
Disaster-Preparedness.
</td>
<td class="bdbstorage">
Leverage the famous robustness, performance, and scalability of BerkeleyDB.
</td>
<td class="ape">
Apelib bridges the gap between ZODB and relational data storage.
It lets developers store ZODB objects in arbitrary databases and
arbitrary formats, without changing application code. It combines
the advantages of orthogonal persistence with relational storage.
</td>
</tr>
<tr><th colspan=4 class=title>Variants</th></tr>
<tr>
<td class="filestorage">
Many people have hacked on FileStorage for their own purposes.
<a href="http://www.zope.org/Members/hathawsh/PartitionedFileStorage">PartitionedFileStorage</a> by Shane Hathaway splits
Data.fs into chunks, for easier manageability.</a>
</td>
<td class="dirstorage">
DirectoryStorage.Full is the only general-purpose production-quality variant.
</td>
<td class="bdbstorage">
bsddb3Storage.Full supports undo and versions, while bsddb3Storage.Minimal does not.
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>External Dependencies</th></tr>
<tr>
<td class="filestorage">
If necessary, a filesystem (plus backup tools, etc) that can deal with one huge Data.fs files. Some systems still have
trouble at the 2G limit.<br>
</td>
<td class="dirstorage">
A robust filesystem that will not dump files in lost+found after a power loss, and which can handle bucketloads
of small files. The developers use reiserfs on linux.
</td>
<td class="bdbstorage">
Sleepycat's Berkeley DB 4.0.14 and Robin Dunn's PyBSDDB 3.4 Python bindings.
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Supports Undo</th></tr>
<tr>
<td class="filestorage">
Yes
</td>
<td class="dirstorage">
Yes
</td>
<td class="bdbstorage">
Yes, in Full variant.
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Supports Versions</th></tr>
<tr>
<td class="filestorage">
Yes
</td>
<td class="dirstorage">
No
</td>
<td class="bdbstorage">
Yes, in Full variant.
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Performance</th></tr>
<tr>
<td class="filestorage">
Blindingly fast. Nothing beats FileStorage, so long as its index fits comfortably in memory.
</td>
<td class="dirstorage">
A little slower than BDBStorage, but not enough to be significant except in a benchmark.
</td>
<td class="bdbstorage">
A little slower than FileStorage, but not enough to be significant except in a benchmark.
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Memory Requirements</th></tr>
<tr>
<td class="filestorage">
It needs to keep its index in memory, which is proportional to storage size. As a general rule,
allow between 2% and 10% of the Data.fs size in RAM or performance will slow to a crawl.
</td>
<td class="dirstorage">
Small, and independant of storage size.
</td>
<td class="bdbstorage">
Small, and independant of storage size.
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Disk Space Efficiency</th></tr>
<tr>
<td class="filestorage">
Data.fs is the benchmark for disk space. During packing it uses double the disk space,
as a new file is built in Data.fs.new.
</td>
<td class="dirstorage">
Roughly 30% more space than Data.fs.
</td>
<td class="bdbstorage">
?
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Packing?</th></tr>
<tr>
<td class="filestorage">
Yes, needs occasional packing to remove old revisions, and unreferenced
objects.
</td>
<td class="dirstorage">
Like FileStorage, but slower. DirectoryStorage has a fail-safe
packing implementation that prevents a packing bug causing permanent data loss.
(The history of FileStorage shows that packing is error-prone)</li>
</td>
<td class="bdbstorage">
Reference counts are used to delete unreferenced objects, except
when there are cycles.
Full needs packing to remove old revisions and cycles of unreferenced objects.
Minimal just for cycles. This packing can be performed automatically.
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Online Backup</th></tr>
<tr>
<td class="filestorage">
Not as standard. You can find some hairy home-brew scripts in the zodb-dev list archives.
</td>
<td class="dirstorage">
Yes, using a standard script with a tar backend. Incremental backups to tar.gz files.
</td>
<td class="bdbstorage">
Yes, as Berkeley DB.
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Online Replication</th></tr>
<tr>
<td class="filestorage">
No.
</td>
<td class="dirstorage">
Yes, A standard script to replicate changes into an identical storage directory
on another machine.
</td>
<td class="bdbstorage">
No. (or does BDB help out here?)
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Storage Checking Tools</th></tr>
<tr>
<td class="filestorage">
A variety of tools in variety of places, for checking and fixing certain types of corruption.
</td>
<td class="dirstorage">
checkds.py, a unified checking tool. Can be run without shutting down the storage.
</td>
<td class="bdbstorage">
No (?)
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Storage Dumping Tools</th></tr>
<tr>
<td class="filestorage">
fsdump.dy
</td>
<td class="dirstorage">
dumpdsf.py
</td>
<td class="bdbstorage">
No (?)
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Startup/Shutdown time</th></tr>
<tr>
<td class="filestorage">
On shutdown the index is written to disk, and reloaded on startup. This takes time proportional to storage size,
but is quick. After an unclean shutdown it needs to rebuild the index, which is slow.</td>
<td class="dirstorage">
Small, and independant of storage size.
</td>
<td class="bdbstorage">
Small, and independant of storage size.
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Recovery after unexpected power loss</th></tr>
<tr>
<td class="filestorage">
Automatically truncates the file to the last complete transaction, and copies
the tail in a file for later inspection.<br>
PartitionedFileStorage may need some manual tweaking, if you are very unlucky.
</td>
<td class="dirstorage">
Automatically deletes any partial transactions.
</td>
<td class="bdbstorage">
Automatically deletes any partial transactions.
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Administrative Overhead</th></tr>
<tr>
<td class="filestorage">
None
</td>
<td class="dirstorage">
None, after the seperate download and install.<br>
</td>
<td class="bdbstorage">
As Berkeley DB. Learn the sleepycat tools.
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Lines of Code (wc -l, excluding tests)</th></tr>
<tr>
<td class="filestorage">
2500, excluding tools
</td>
<td class="dirstorage">
4800, including tools
</td>
<td class="bdbstorage">
3500
</td>
<td class="ape">
7000
</td>
</tr>
<tr><th colspan=4 class=title>Other Features</th></tr>
<tr>
<td class="filestorage">
None
</td>
<td class="dirstorage">
Automatically checks for dangling references on commit, to avoid several possible
causes of POSKeyError exceptions.
</td>
<td class="bdbstorage">
None
</td>
<td class="ape">
?
</td>
</tr>
<tr><th colspan=4 class=title>Missing Features</th></tr>
<tr>
<td class="filestorage">
None
</td>
<td class="dirstorage">
Does not report storage size or number of objects.
</td>
<td class="bdbstorage">
None
</td>
<td class="ape">
?
</td>
</tr>
</table>
</body>
</html>