hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11397) When merging expired stripes, we need to create an empty file to preserve metadata.
Date Wed, 25 Jun 2014 04:17:24 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14043020#comment-14043020

Hadoop QA commented on HBASE-11397:

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  against trunk revision .
  ATTACHMENT ID: 12652330

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 3 new or modified

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:red}-1 findbugs{color}.  The patch appears to introduce 2 new Findbugs (version
1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9842//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9842//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9842//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9842//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9842//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9842//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9842//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9842//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9842//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9842//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9842//console

This message is automatically generated.

> When merging expired stripes, we need to create an empty file to preserve metadata.
> -----------------------------------------------------------------------------------
>                 Key: HBASE-11397
>                 URL: https://issues.apache.org/jira/browse/HBASE-11397
>             Project: HBase
>          Issue Type: Bug
>          Components: Compaction
>    Affects Versions: 0.98.2
>         Environment: jdk1.7.0_45, hadoop-cdh5, hbase-0.98.2
>            Reporter: Victor Xu
>            Assignee: Victor Xu
>             Fix For: 0.99.0, 0.98.4
>         Attachments: 11397-v3.patch, HBASE-11397-AssertionError.png, HBASE-11397-HDFS.png,
HBASE-11397-RS-Log.png, HBASE-11397-Stripe-Info.png, HBASE-11397-test.patch, HBASE-11397-v2.patch,
> Stripe Compaction is a good feature in 0.96 and 0.98. But when I used it in a heavy-write
non-uniform row keys scenario(e.g. time dimension in a key), I came across some problems.

> I made my stripes split at the size of 2G(hbase.store.stripe.sizeToSplit=2G), and soon
there were tens of them. It was true that only the last stripe receiving the new keys kept
compacting - old data didn't compact as much, or at all. However, the old stripes were still
there when they all expired. I checked the source code and found that when compacting expired
stripes, the StoreScanner may return no KVs so that SizeMultiWriter.append() is never called.
> My solution is to create an empty file to preserve metadata at the end of the SizeMultiWriter.commitWritersInternal().

This message was sent by Atlassian JIRA

View raw message