Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 29 Jan 2013 16:39:18 +0000 (UTC)
From: "Hadoop QA (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12624812.1356009924397.209001.1359477558722@arcas>
In-Reply-To: <JIRA.12624812.1356009924397@arcas>
References: <JIRA.12624812.1356009924397@arcas>
Subject: [jira] [Commented] (HBASE-7403) Online Merge
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HBASE-7403?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1356=
5502#comment-13565502 ]=20

Hadoop QA commented on HBASE-7403:
----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest a=
ttachment=20
  http://issues.apache.org/jira/secure/attachment/12566995/hbase-7403-trunk=
v12.patch
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author=
 tags.

    {color:green}+1 tests included{color}.  The patch appears to include 6 =
new or modified tests.

    {color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoo=
p 2.0 profile.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any =
warning messages.

    {color:green}+1 javac{color}.  The applied patch does not increase the =
total number of javac compiler warnings.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new =
Findbugs (version 1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not incre=
ase the total number of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces lines longer th=
an 100

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.master.TestCatalogJanitor

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4234//tes=
tReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4234=
//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4234=
//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4234=
//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.=
html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4234=
//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4234=
//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4234=
//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.h=
tml
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4234//c=
onsole

This message is automatically generated.
               =20
> Online Merge
> ------------
>
>                 Key: HBASE-7403
>                 URL: https://issues.apache.org/jira/browse/HBASE-7403
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.94.3
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>             Fix For: 0.96.0, 0.94.6
>
>         Attachments: 7403-trunkv5.patch, 7403-trunkv6.patch, 7403v5.diff,=
 7403-v5.txt, 7403v5.txt, hbase-7403-94v1.patch, hbase-7403-trunkv10.patch,=
 hbase-7403-trunkv11.patch, hbase-7403-trunkv12.patch, hbase-7403-trunkv1.p=
atch, hbase-7403-trunkv5.patch, hbase-7403-trunkv6.patch, hbase-7403-trunkv=
7.patch, hbase-7403-trunkv8.patch, hbase-7403-trunkv9.patch, merge region.p=
df
>
>
> The feature of this online merge:
> 1.Online,no necessary to disable table
> 2.Less change for current code, could applied in trunk,0.94 or 0.92,0.90
> 3.Easy to call merege request, no need to input a long region name, only =
encoded name enough
> 4.No limit when operation, you don't need to tabke care the events like S=
erver Dead, Balance, Split, Disabing/Enabing table, no need to take care wh=
ether you send a wrong merge request, it has alread done for you
> 5.Only little offline time for two merging regions
> Usage:
> 1.Tool: =20
> bin/hbase org.apache.hadoop.hbase.util.OnlineMerge [-force] [-async] [-sh=
ow] <table-name> <region-encodedname-1> <region-encodedname-2>
> 2.API: static void MergeManager#createMergeRequest
> We need merge in the following cases=EF=BC=9A
> 1.Region hole or region overlap, can=E2=80=99t be fix by hbck
> 2.Region become empty because of TTL and not reasonable Rowkey design
> 3.Region is always empty or very small because of presplit when create ta=
ble
> 4.Too many empty or small regions would reduce the system performance(e.g=
. mslab)
> Current merge tools only support offline and are not able to redo if exce=
ption is thrown in the process of merging, causing a dirty data
> For online system, we need a online merge.
> This implement logic of this patch for  Online Merge is :
> For example, merge regionA and regionB into regionC
> 1.Offline the two regions A and B
> 2.Merge the two regions in the HDFS(Create regionC=E2=80=99s directory, m=
ove regionA=E2=80=99s and regionB=E2=80=99s file to regionC=E2=80=99s direc=
tory, delete regionA=E2=80=99s and regionB=E2=80=99s directory)
> 3.Add the merged regionC to .META.
> 4.Assign the merged regionC
> As design of this patch , once we do the merge work in the HDFS,we could =
redo it until successful if it throws exception or abort or server restart,=
 but couldn=E2=80=99t be rolled back.=20
> It depends on
> Use zookeeper to record the transaction journal state, make redo easier
> Use zookeeper to send/receive merge request
> Merge transaction is executed on the master
> Support calling merge request through API or shell tool
> About the merge process, please see the attachment and patch

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrato=
rs
For more information on JIRA, see: http://www.atlassian.com/software/jira