hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-1186) Memory-aware Maps with LRU eviction
Date Thu, 05 Feb 2009 17:57:59 GMT
Memory-aware Maps with LRU eviction

                 Key: HBASE-1186
                 URL: https://issues.apache.org/jira/browse/HBASE-1186
             Project: Hadoop HBase
          Issue Type: New Feature
            Reporter: Jonathan Gray
            Priority: Critical
             Fix For: 0.20.0

Caching is key for 0.20.  We need a set of memory-aware data structures to manage our caches.

I propose two initial classes:  LruHashMap and LruBlockMap

*LruHashMap* is currently being used over in HBASE-80 for the Cell cache.  Erik Holstad has
done extensive testing and benchmarking and will post results over in this issue.
- Memory-aware
- Fixed size
- LRU eviction

*LruBlockMap* can be used for the block caching of the new file format in HBASE-61.  It should
try to use all available memory, but must contend with Memcaches so is resizable to deal with
heap pressure.  Adding high priority blocks (evicted last) gives us in-memory functionality
as described in bigtable paper.
- Memory-aware
- Fully resizable
- LRU eviction (with some additions)
- High priority blocks
- _Optional: Scan resistant algorithm_

Part of this issue is also solving how we will determine the size of cached objects.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message