hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11397) When merging expired stripes, we need to create an empty file to preserve metadata.
Date Mon, 23 Jun 2014 00:00:30 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14040308#comment-14040308

Ted Yu commented on HBASE-11397:

Please generate the patch from root of workspace.
i.e. from hbase/hbase-0.98-etao/branches/hbase-0.98.2-etao

> When merging expired stripes, we need to create an empty file to preserve metadata.
> -----------------------------------------------------------------------------------
>                 Key: HBASE-11397
>                 URL: https://issues.apache.org/jira/browse/HBASE-11397
>             Project: HBase
>          Issue Type: Bug
>          Components: Compaction
>    Affects Versions: 0.98.2
>         Environment: jdk1.7.0_45, hadoop-cdh5, hbase-0.98.2
>            Reporter: Victor Xu
>         Attachments: HBASE-11397-AssertionError.png, HBASE-11397-HDFS.png, HBASE-11397-RS-Log.png,
HBASE-11397-Stripe-Info.png, HBASE-11397.patch
> Stripe Compaction is a good feature in 0.96 and 0.98. But when I used it in a heavy-write
non-uniform row keys scenario(e.g. time dimension in a key), I came across some problems.

> I made my stripes split at the size of 2G(hbase.store.stripe.sizeToSplit=2G), and soon
there were tens of them. It was true that only the last stripe receiving the new keys kept
compacting - old data didn't compact as much, or at all. However, the old stripes were still
there when they all expired. I checked the source code and found that when compacting expired
stripes, the StoreScanner may return no KVs so that SizeMultiWriter.append() is never called.
> My solution is to create an empty file to preserve metadata at the end of the SizeMultiWriter.commitWritersInternal().

This message was sent by Atlassian JIRA

View raw message