New compression functionality in ovdb

Heath Kehoe hakehoe at avalon.net
Sat Jun 12 19:40:07 UTC 2004


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I've just committed to CURRENT new code that enables compression
of overview records in ovdb.  So far, my testing has shown that
this compression is a big win for ovdb, mostly because the average
record size is smaller which increases the average keys-per-page
in the BerkeleyDB Btree databases, thus improving performance
(see http://www.sleepycat.com/docs/ref/am_conf/intro.html ).
Only records that are longer than 600 bytes get compressed.

The change is made in a way to insure minimal disruption.
Compression defaults to off, and must be enabled using the
'compress' setting in ovdb.conf.  You can enable compression
without having to rebuild the database, in which case new records
will be compressed but existing records will remain the same.
To gain immediate benefit, you'll have to delete the databases
and rebuild them using makehistory.

If you don't enable compression the database will remain
backward-compatible with older versions of ovdb.

On my server with about 21 million overview records, after
rebuilding the databases with compression enabled, the overview
directory went from around 20 GB to 12.6 GB, and the expireover
run went from 6 hours to 2 hours.

I also have this working on a 2.4.1 server, so if anyone wants
to try compression I can make a patch available on my web page.

- - heath
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (Darwin)

iD8DBQFAy1wd4uXPAG0A1J4RAgGvAJ9CbyNYFyx+fITQpPkmweGj7YlhXQCgusla
HuBY//nGdrI6mJ42oxT9z80=
=qK1r
-----END PGP SIGNATURE-----



More information about the inn-workers mailing list