Optimizing WCI

Optimizing of Query Plan

Optimizing of Data Reads

Distributed Database Cluster

Run ANALYZE manually after loading operations

Normally ANALYZE is run with the VACUUM ANALYZE, but this may take a while (VACUUM takes a long time). Running ANALYZE immediately cuts down on that time.

memcache the WCI backend

The performance data we are seeing indicate that the database is not effectively using the memory it has assigned. We see no significant difference, for instance, between performance of a machine with 8GB and one with 16GB of RAM. The idea here would be - basically - to force reuse of RAM for the database installation. A suggested implementation is given below:

Set up memcached (or an alternatively main-memory system, but memcached is probably best).

Hash Large Object segments into memcached. Since our LOs are immutable (at least in theory - we need to do something about this in practice for WDB), we can simply implement the following for wci.read:

If OID in memcached, retrieve object from memcached and extract values

If OID is not memcached, retrieve object from database - extract data and cache OID

Postgres support for memcached exists: pgmemcache

memcached exists in debian