Optimizing WCI

The performance data we are seeing indicate that the database is not effectively using the memory it has assigned. We see no significant difference, for instance, between performance of a machine with 8GB and one with 16GB of RAM. The idea here would be - basically - to force reuse of RAM for the database installation. A suggested implementation is given below:

• Set up memcached (or an alternatively main-memory system, but memcached is probably best).
• Hash Large Object segments into memcached. Since our LOs are immutable (at least in theory - we need to do something about this in practice for WDB), we can simply implement the following for wci.read:
  • If OID in memcached, retrieve object from memcached and extract values
  • If OID is not memcached, retrieve object from database - extract data and cache OID

Postgres support for memcached exists: pgmemcache. memcached exists in debian.

Pros: Will likely reduce dependency on disk speed even further, since a large proportion of the queries hit the same OID's repeatedly.

Cons: Currently (Dec 2008), WDB appears to be CPU-bound for large workloads. Adding memcached does not solve these problems.