hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom White (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1398) Add in-memory caching of data
Date Thu, 17 Jan 2008 21:47:34 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12560091#action_12560091
] 

Tom White commented on HADOOP-1398:
-----------------------------------

I'm trying to add a new parameter to HColumnDescriptor and would appreciate a little guidance.
Do I need to worry about the version number? Is the order of the serialized fields important?
It would be nice to group together the caching related ones if possible, so the block cache
parameter would naturally sit next to the inMemory one. Ditto for the Thrift representation
- how does it handle versioning? Thanks.

> Add in-memory caching of data
> -----------------------------
>
>                 Key: HADOOP-1398
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1398
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: contrib/hbase
>            Reporter: Jim Kellerman
>            Priority: Trivial
>         Attachments: hadoop-blockcache-v2.patch, hadoop-blockcache.patch
>
>
> Bigtable provides two in-memory caches: one for row/column data and one for disk block
caches.
> The size of each cache should be configurable, data should be loaded lazily, and the
cache managed by an LRU mechanism.
> One complication of the block cache is that all data is read through a SequenceFile.Reader
which ultimately reads data off of disk via a RPC proxy for ClientProtocol. This would imply
that the block caching would have to be pushed down to either the DFSClient or SequenceFile.Reader

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message