hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-11397) When merging expired stripes, we need to create an empty file to preserve metadata.
Date Wed, 25 Jun 2014 02:31:25 GMT

     [ https://issues.apache.org/jira/browse/HBASE-11397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Ted Yu updated HBASE-11397:

    Attachment: 11397-v3.patch

The import portion of the test patch had some conflicts.

Patch v3 combines v2 with test patch.

> When merging expired stripes, we need to create an empty file to preserve metadata.
> -----------------------------------------------------------------------------------
>                 Key: HBASE-11397
>                 URL: https://issues.apache.org/jira/browse/HBASE-11397
>             Project: HBase
>          Issue Type: Bug
>          Components: Compaction
>    Affects Versions: 0.98.2
>         Environment: jdk1.7.0_45, hadoop-cdh5, hbase-0.98.2
>            Reporter: Victor Xu
>            Assignee: Victor Xu
>             Fix For: 0.99.0, 0.98.4
>         Attachments: 11397-v3.patch, HBASE-11397-AssertionError.png, HBASE-11397-HDFS.png,
HBASE-11397-RS-Log.png, HBASE-11397-Stripe-Info.png, HBASE-11397-test.patch, HBASE-11397-v2.patch,
> Stripe Compaction is a good feature in 0.96 and 0.98. But when I used it in a heavy-write
non-uniform row keys scenario(e.g. time dimension in a key), I came across some problems.

> I made my stripes split at the size of 2G(hbase.store.stripe.sizeToSplit=2G), and soon
there were tens of them. It was true that only the last stripe receiving the new keys kept
compacting - old data didn't compact as much, or at all. However, the old stripes were still
there when they all expired. I checked the source code and found that when compacting expired
stripes, the StoreScanner may return no KVs so that SizeMultiWriter.append() is never called.
> My solution is to create an empty file to preserve metadata at the end of the SizeMultiWriter.commitWritersInternal().

This message was sent by Atlassian JIRA

View raw message