Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <7546205.1200916176318.JavaMail.jira@brutus>
Date: Mon, 21 Jan 2008 03:49:36 -0800 (PST)
From: "Tom White (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Updated: (HADOOP-1398) Add in-memory caching of data
In-Reply-To: <28911305.1179772096114.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HADOOP-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tom White updated HADOOP-1398:
------------------------------

    Attachment: hadoop-blockcache-v4.patch

bq. In the below from HStoreFile, blockCacheEnabled method argument is not being passed to the MapFile constructors.

Thanks - this had the effect of never enabling the cache! I've fixed this.

bq. Out of interest, did you regenerate the thrift or hand-edit it? Changes look right - just wondering.

I regenerated using the latest thrift trunk.

bq. Default ReferenceMap constructor makes for hard keys and soft values. If value has been let go by the GC, does the corresponding key just stay in the Map?

Yes - I checked the source.

This patch also includes changes to HBase Shell so you can alter a table to enable block caching.

> Add in-memory caching of data
> -----------------------------
>
>                 Key: HADOOP-1398
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1398
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: contrib/hbase
>            Reporter: Jim Kellerman
>            Priority: Trivial
>         Attachments: commons-collections-3.2.jar, hadoop-blockcache-v2.patch, hadoop-blockcache-v3.patch, hadoop-blockcache-v4.patch, hadoop-blockcache.patch
>
>
> Bigtable provides two in-memory caches: one for row/column data and one for disk block caches.
> The size of each cache should be configurable, data should be loaded lazily, and the cache managed by an LRU mechanism.
> One complication of the block cache is that all data is read through a SequenceFile.Reader which ultimately reads data off of disk via a RPC proxy for ClientProtocol. This would imply that the block caching would have to be pushed down to either the DFSClient or SequenceFile.Reader

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.