Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 9 Sep 2011 20:48:11 +0000 (UTC)
From: "jiraposter@reviews.apache.org (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: 
 <921987920.9889.1315601291421.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <199150873.5582.1315527008794.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HBASE-4358) Batch Table Alter Operations
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HBASE-4358?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1310=
1551#comment-13101551 ]=20

jiraposter@reviews.apache.org commented on HBASE-4358:
------------------------------------------------------


bq.  On 2011-09-09 19:48:45, Ted Yu wrote:
bq.  > /src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamily=
Handler.java, line 64
bq.  > <https://reviews.apache.org/r/1768/diff/1/?file=3D38941#file38941lin=
e64>
bq.  >
bq.  >     After HBASE-451, changes to table descriptor have to be persiste=
d to HDFS.
bq.  >    =20
bq.  >     I browsed handleTableOperation() methods in this patch and didn'=
t find that.

I'm not familiar enough with how exactly table descriptors are persisted to=
 be able to tell you for certain that this approach correctly ensures persi=
stence. But I can confidently tell you that this diff does everything that =
the current trunk does with regards to updating table descriptors. If you l=
ook at the actual implementation of MasterFileSystem.{add,modify,Delete}Col=
umn(), it gets a table descriptor from master services, modifies the table =
descriptor appropriately, then adds it back to master services' table descr=
iptors. Between handleTableOperation() and updateTableDescriptor(), this pa=
tch follows the exact same procedure, and has the same instance of MasterSe=
rvices. This separation is for the purpose of enabling batching to happen i=
n a way that doesn't leave the system in an intermediate state in the case =
of a thrown exception. From basic testing, I see that the table descriptor =
changes are actually being written to my fs. If this procedure is not enoug=
h to actually ensure that this happens, we should file a separate JIRA to l=
ook into it.


- Riley


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1768/#review1843
-----------------------------------------------------------


On 2011-09-09 18:39:05, Riley Patterson wrote:
bq. =20
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/1768/
bq.  -----------------------------------------------------------
bq. =20
bq.  (Updated 2011-09-09 18:39:05)
bq. =20
bq. =20
bq.  Review request for hbase.
bq. =20
bq. =20
bq.  Summary
bq.  -------
bq. =20
bq.  Currently, the RPC provides no way of asking for several table alterat=
ions at once, and the master has no way of batch handling alter requests. T=
hus, when the user requests several changes at the same time (i.e. add thes=
e I columns, delete these J columns, and modify these K columns), each regi=
on is brought down (I+J+K) times so that it can reflect the new schema. Add=
itionally, multiple writes are made to META, and multiple RPC calls must be=
 made.
bq. =20
bq.  This patch provides batching for these operations, both at the RPC lev=
el and within the Master's TableEventHandlers. This involves a bit of reorg=
anization in the TableEventHandler class hierarchy, and a new TableEventHan=
dler, TableMultiFamilyHandler. The net effect ends up being the difference =
seen here:
bq. =20
bq.  Before patch:
bq.  hbase(main):001:0> alter 'peeps', {NAME =3D> 'rawr'}, {METHOD =3D> 'de=
lete', NAME =3D> 'name'}
bq.  Updating all regions with the new schema...
bq.  1/1 regions updated.
bq.  Done.
bq.  Updating all regions with the new schema...
bq.  1/1 regions updated.
bq.  Done.
bq.  0 row(s) in 2.6450 seconds
bq. =20
bq.  After patch:
bq.  hbase(main):002:0> alter 'peeps', {NAME =3D> 'rawr'}, {METHOD =3D> 'de=
lete', NAME =3D> 'name'}
bq.  Updating all regions with the new schema...
bq.  1/1 regions updated.
bq.  Done.
bq.  0 row(s) in 1.1930 seconds
bq. =20
bq.  Regions are only brought down once, and the duration is cut 1/N.
bq. =20
bq. =20
bq.  This addresses bug HBASE-4358.
bq.      https://issues.apache.org/jira/browse/HBASE-4358
bq. =20
bq. =20
bq.  Diffs
bq.  -----
bq. =20
bq.    /src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java 116693=
3=20
bq.    /src/main/java/org/apache/hadoop/hbase/coprocessor/BaseMasterObserve=
r.java 1166933=20
bq.    /src/main/java/org/apache/hadoop/hbase/coprocessor/MasterObserver.ja=
va 1166933=20
bq.    /src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java 11=
66933=20
bq.    /src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java 116=
6933=20
bq.    /src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1166933=
=20
bq.    /src/main/java/org/apache/hadoop/hbase/master/MasterCoprocessorHost.=
java 1166933=20
bq.    /src/main/java/org/apache/hadoop/hbase/master/handler/TableAddFamily=
Handler.java 1166933=20
bq.    /src/main/java/org/apache/hadoop/hbase/master/handler/TableDeleteFam=
ilyHandler.java 1166933=20
bq.    /src/main/java/org/apache/hadoop/hbase/master/handler/TableFamilyHan=
dler.java PRE-CREATION=20
bq.    /src/main/java/org/apache/hadoop/hbase/master/handler/TableModifyFam=
ilyHandler.java 1166933=20
bq.    /src/main/java/org/apache/hadoop/hbase/master/handler/TableMultiFami=
lyHandler.java PRE-CREATION=20
bq.    /src/main/ruby/hbase/admin.rb 1166933=20
bq.    /src/test/java/org/apache/hadoop/hbase/coprocessor/TestMasterObserve=
r.java 1166933=20
bq. =20
bq.  Diff: https://reviews.apache.org/r/1768/diff
bq. =20
bq. =20
bq.  Testing
bq.  -------
bq. =20
bq.  Sanity checked functionality in psuedo-distributed mode (tried several=
 permutations of different alterations, all completed successfully and with=
 only one round of region restarts). Ran all unit tests successfully.
bq. =20
bq. =20
bq.  Thanks,
bq. =20
bq.  Riley
bq. =20
bq.


> Batch Table Alter Operations
> ----------------------------
>
>                 Key: HBASE-4358
>                 URL: https://issues.apache.org/jira/browse/HBASE-4358
>             Project: HBase
>          Issue Type: Improvement
>          Components: ipc, master, shell
>    Affects Versions: 0.92.0
>            Reporter: Riley Patterson
>            Assignee: Riley Patterson
>            Priority: Minor
>         Attachments: HBASE-4358.patch
>
>
> Currently, the RPC provides no way of asking for several table alteration=
s at once, and the master has no way of batch handling alter requests. Thus=
, when the user requests several changes at the same time (i.e. add these I=
 columns, delete these J columns, and modify these K columns), each region =
is brought down (I+J+K) times so that it can reflect the new schema. Additi=
onally, multiple writes are made to META, and multiple RPC calls must be ma=
de.
> This patch provides batching for these operations, both at the RPC level =
and within the Master's TableEventHandlers. This involves a bit of reorgani=
zation in the TableEventHandler class hierarchy, and a new TableEventHandle=
r, TableMultiFamilyHandler. The net effect ends up being the difference see=
n here:
> Before patch:
> hbase(main):001:0> alter 'peeps', {NAME =3D> 'rawr'}, {METHOD =3D> 'delet=
e', NAME =3D> 'name'}
> Updating all regions with the new schema...
> 1/1 regions updated.
> Done.
> Updating all regions with the new schema...
> 1/1 regions updated.
> Done.
> 0 row(s) in 2.6450 seconds
> After patch:
> hbase(main):002:0> alter 'peeps', {NAME =3D> 'rawr'}, {METHOD =3D> 'delet=
e', NAME =3D> 'name'}
> Updating all regions with the new schema...
> 1/1 regions updated.
> Done.
> 0 row(s) in 1.1930 seconds
> Regions are only brought down once, and the duration is cut 1/N.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira