jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Mueller <muel...@adobe.com>
Subject Re: ids and locality
Date Wed, 12 Jan 2011 11:36:52 GMT

>If the memory cost is too big, you could also use an LRU mechanism to
>only keep the most recently accessed identifiers indexed in memory.

That would increase the memory footprint of the index significantly. Also,
maintaining the LRU cache on an item / entry level is relatively slow.
Instead, I would cache index _pages_ (or index 'blocks').

>I think an append-only or at least journaled storage is pretty much a
>requirement for any modern media that benefits from locality of access
>(and thus is relevant to this discussion), so it should be no problem
>to write incremental index updates only occasionally.

For Jackrabbit 3, I wouldn't bet everything on append-only storage. I
think we should support relational databases, in-memory repositories, and
append-only storage. Relational databases, because this is the only viable
option for some (due to business, hardware or software restrictions - I
would like to see Jackrabbit 3 on Android using SQLite). In-memory
repositories for testing and specialized use cases. Append-only storage
for performance and simplicity.

For certain storage systems, locality of data access (accessing the data)
is not as important, for example for solid state disks. But I don't think
we can bet on SSDs currently.


View raw message