On Feb 2, 2008 1:03 PM, Howard Chu <hyc@symas.com> wrote:
> Very good point about these opposing factors. So I figure OpenLDAP just
> uses the cache in the underlying B-Tree instead of managing some kind of
> separate entry cache?

No, we have a separate entry cache too. I originally wrote back-bdb without
any entry caching, but Jong @ IBM benchmarked/profiled it and implemented an
entry cache for it before 2.1 was released. (Back then I was happy just to get
it working, at speeds comparable to back-ldbm. How things change ;)

The motivation here is that the serialized data that we store in the database
is not directly usable with our in-memory data structures. As such, even when
the B-Tree cache is 100% effective, there's a cost associated with
deserialization that we can avoid by using our own entry cache.

That's exactly the same problem we have: deserialization + not being able to manipulate the raw binary data in accordance with internal data structures.  It's good to get this feedback thanks.  This is why we have an entry cache although there are some nasty issues we have to contend with like memory management and sharing a single cache across our partitions instead of each partition managing it's own entry cache.  Unfortunately because of the way we designed search capabilities into the partition itself we cannot have it share a global cache. 
 
There are so many problems to solve, we're just scratching the surface now as we make our way towards 2.0.  IMO 1.0 was a toy.  2.0 has a chance to solve some neat problems and actually put several ideas to the test. 


There are plenty of interesting data structures/system architectures to
explore, and now you've got the lab resources to try them out.

Yes I am looking forward to that: it will be fun.  I think the first thing I want to do is move to MINA 2.0 and try the server out with the APR transport and the NIO transport to compare the results.

/me salivating now
 

>      > Yeah they're great ideas. We just need to have a solid SLAMD lab and
>      > start testing these ideas out. I got the machines:
>      >
>      > 9 load injectos
>      > 1 SLAMD Server
>      > 1 beefy server for running ApacheDS>

Thanks again,
Alex