hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12583) Allow creating reference files even the split row not lies in the storefile range if required
Date Tue, 09 Dec 2014 15:13:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239521#comment-14239521
] 

Hadoop QA commented on HBASE-12583:
-----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12686006/HBASE-12583_v3.patch
  against master branch at commit 8a2c84156a2829f44307e5aff07323902580af86.
  ATTACHMENT ID: 12686006

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 6 new or modified
tests.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 checkstyle{color}.  The applied patch does not increase the total number
of checkstyle errors

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
2.0.3) warnings.

    {color:red}-1 release audit{color}.  The applied patch generated 1 release audit warnings
(more than the master's current 0 warnings).

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//artifact/patchprocess/patchReleaseAuditWarnings.txt
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//artifact/patchprocess/checkstyle-aggregate.html

  Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/12015//console

This message is automatically generated.

> Allow creating reference files even the split row not lies in the storefile range if
required
> ---------------------------------------------------------------------------------------------
>
>                 Key: HBASE-12583
>                 URL: https://issues.apache.org/jira/browse/HBASE-12583
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: rajeshbabu
>            Assignee: rajeshbabu
>              Labels: Phoenix
>             Fix For: 1.0.0, 2.0.0, 0.98.9
>
>         Attachments: HBASE-12583.patch, HBASE-12583_98.patch, HBASE-12583_98_v2.patch,
HBASE-12583_branch1.patch, HBASE-12583_branch1_v2.patch, HBASE-12583_v2.patch, HBASE-12583_v3.patch
>
>
> Currently in HRegionFileSystem#splitStoreFile we are not creating reference files if
the split row not lies in the storefile range that means one of the child region doesn't have
any data.
> {code}
>    // Check whether the split row lies in the range of the store file
>     // If it is outside the range, return directly.
>     if (top) {
>       //check if larger than last key.
>       KeyValue splitKey = KeyValueUtil.createFirstOnRow(splitRow);
>       byte[] lastKey = f.createReader().getLastKey();
>       // If lastKey is null means storefile is empty.
>       if (lastKey == null) return null;
>       if (f.getReader().getComparator().compareFlatKey(splitKey.getBuffer(),
>           splitKey.getKeyOffset(), splitKey.getKeyLength(), lastKey, 0, lastKey.length)
> 0) {
>         return null;
>       }
>     } else {
>       //check if smaller than first key
>       KeyValue splitKey = KeyValueUtil.createLastOnRow(splitRow);
>       byte[] firstKey = f.createReader().getFirstKey();
>       // If firstKey is null means storefile is empty.
>       if (firstKey == null) return null;
>       if (f.getReader().getComparator().compareFlatKey(splitKey.getBuffer(),
>           splitKey.getKeyOffset(), splitKey.getKeyLength(), firstKey, 0, firstKey.length)
< 0) {
>         return null;
>       }
>     }
> {code}
> In some cases when split row should be compared with part of rowkey(in composite rowkey)
mainly for secondary index tables we need to create reference files even when split row not
lies in the storefile range so that they can be rewritten to it's child regions by some custom
half store file reader which compare the part of row key with split row.
> The check of comparing split row with storefile range and returning directly can be avoided
by having special boolean attribute in table descriptor when it set to true. Or else we can
have coprocessor hooks so that in the hooks we can create the references and bypass.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message