hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13031) Ability to snapshot based on a key range
Date Mon, 02 Mar 2015 20:32:06 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14343709#comment-14343709

Hadoop QA commented on HBASE-13031:

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  against master branch at commit 7b5c9eabacf5019d5b6aba95ba5a4fcb7dc8d8e5.
  ATTACHMENT ID: 12701930

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 12 new or modified
    {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions
(2.4.1 2.5.2 2.6.0)

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

                {color:red}-1 checkstyle{color}.  The applied patch generated 1938 checkstyle
errors (more than the master's current 1937 errors).

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
2.0.3) warnings.

    {color:red}-1 release audit{color}.  The applied patch generated 1 release audit warnings
(more than the master's current 0 warnings).

    {color:red}-1 lineLengths{color}.  The patch introduces the following lines longer than
    +              new java.lang.String[] { "Name", "Table", "CreationTime", "Type", "Version",
"Owner", "StartRow", "StopRow", });
+      if (metaChanges.hasRegionsToRemove() && !ClientSnapshotDescriptionUtils.isPartialSnapshot(snapshot))
+          start = org.apache.hadoop.hbase.util.Bytes.toStringBinary(snapshot.getStartRow.toByteArray)
+          formatter.row([ snapshot.getName, snapshot.getTable + " (" + creation_time + ")",
"[ " +  start + " => " + stop + " ]"])
+  hbase> snapshot 'sourceTable', 'snapshotName', {SKIP_FLUSH => true}, {STARTROW =>
'key'}, {STOPROW => 'key'}

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//artifact/patchprocess/patchReleaseAuditWarnings.txt
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//artifact/patchprocess/checkstyle-aggregate.html

                Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/13037//console

This message is automatically generated.

> Ability to snapshot based on a key range
> ----------------------------------------
>                 Key: HBASE-13031
>                 URL: https://issues.apache.org/jira/browse/HBASE-13031
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: churro morales
>            Assignee: churro morales
>             Fix For: 2.0.0, 0.94.26, 1.1.0, 0.98.11
>         Attachments: HBASE-13031-v1.patch, HBASE-13031.patch
> Posted on the mailing list and seems like some people are interested.  A little background
for everyone.
> We have a very large table, we would like to snapshot and transfer the data to another
cluster (compressed data is always better to ship).  Our problem lies in the fact it could
take many weeks to transfer all of the data and during that time with major compactions, the
data stored in dfs has the potential to double which would cause us to run out of disk space.
> So we were thinking about allowing the ability to snapshot a specific key range.  
> Ideally I feel the approach is that the user would specify a start and stop key, those
would be associated with a region boundary.  If between the time the user submits the request
and the snapshot is taken the boundaries change (due to merging or splitting of regions) the
snapshot should fail.
> We would know which regions to snapshot and if those changed between when the request
was submitted and the regions locked, the snapshot could simply fail and the user would try
again, instead of potentially giving the user more / less than what they had anticipated.
 I was planning on storing the start / stop key in the SnapshotDescription and from there
it looks pretty straight forward where we just have to change the verifier code to accommodate
the key ranges.  
> If this design sounds good to anyone, or if I am overlooking anything please let me know.
 Once we agree on the design, I'll write and submit the patches.

This message was sent by Atlassian JIRA

View raw message