hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Created: (HBASE-1186) Memory-aware Maps with LRU eviction
Date Thu, 05 Feb 2009 17:57:59 GMT
Memory-aware Maps with LRU eviction
-----------------------------------

                 Key: HBASE-1186
                 URL: https://issues.apache.org/jira/browse/HBASE-1186
             Project: Hadoop HBase
          Issue Type: New Feature
            Reporter: Jonathan Gray
            Priority: Critical
             Fix For: 0.20.0


Caching is key for 0.20.  We need a set of memory-aware data structures to manage our caches.

I propose two initial classes:  LruHashMap and LruBlockMap

*LruHashMap* is currently being used over in HBASE-80 for the Cell cache.  Erik Holstad has
done extensive testing and benchmarking and will post results over in this issue.
- Memory-aware
- Fixed size
- LRU eviction

*LruBlockMap* can be used for the block caching of the new file format in HBASE-61.  It should
try to use all available memory, but must contend with Memcaches so is resizable to deal with
heap pressure.  Adding high priority blocks (evicted last) gives us in-memory functionality
as described in bigtable paper.
- Memory-aware
- Fully resizable
- LRU eviction (with some additions)
- High priority blocks
- _Optional: Scan resistant algorithm_

Part of this issue is also solving how we will determine the size of cached objects.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message