Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 10 Feb 2012 19:04:59 +0000 (UTC)
From: "Kannan Muthukkaruppan (Commented) (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: 
 <106695969.25339.1328900699713.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <2059322234.68806.1327358140886.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HBASE-5263) Preserving cached data on
 compactions through cache-on-write
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HBASE-5263?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1320=
5645#comment-13205645 ]=20

Kannan Muthukkaruppan commented on HBASE-5263:
----------------------------------------------

Promising idea!=20

In terms of the implementation details, it would be nice to avoid some path=
ological cases... were cold data (which was in the cache but almost on its =
way out of the cache) becomes hot again. I am guessing a naive approach cou=
ld have this pitfall, but something that additionally takes into considerat=
ion the hotness of the keys in the block and appropriately places the data =
in the correct place in the blockcache LRU would not. Haven't thought throu=
gh much about the implementation details... but wanted to throw out the ini=
tial thoughts at least.

See also related idea by Liyin here: HBASE-5263. These could be complementa=
ry approaches.
               =20
> Preserving cached data on compactions through cache-on-write
> ------------------------------------------------------------
>
>                 Key: HBASE-5263
>                 URL: https://issues.apache.org/jira/browse/HBASE-5263
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Mikhail Bautin
>            Assignee: Mikhail Bautin
>            Priority: Minor
>
> We are tackling HBASE-3976 and HBASE-5230 to make sure we don't trash the=
 block cache on compactions if cache-on-write is enabled. However, it would=
 be ideal to reduce the effect compactions have on the cached data. For eve=
ry block we are writing for a compacted file we can decide whether it needs=
 to be cached based on whether the original blocks containing the same data=
 were already in cache. More precisely, for every HFile reader in a compact=
ion we can maintain a boolean flag saying whether the current key-value cam=
e from a disk IO or the block cache. In the HFile writer for the compaction=
's output we can maintain a flag that is set if any of the key-values in th=
e block being written came from a cached block, use that flag at the end of=
 a block to decide whether to cache-on-write the block, and reset the flag =
to false on a block boundary. If such an inclusive approach would still tra=
sh the cache, we could restrict the total number of blocks to be cached per=
 an output HFile, switch to an "and" logic instead of "or" logic for decidi=
ng whether to cache an output file block, or only cache a certain percentag=
e of output file blocks that contain some of the previously cached data.=20
> Thanks to Nicolas for this elegant online algorithm idea!

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrato=
rs: https://issues.apache.org/jira/secure/ContactAdministrators!default.jsp=
a
For more information on JIRA, see: http://www.atlassian.com/software/jira