Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Subject: Review Request: SplitTransaction.splitStoreFiles slows splits a lot
From: "Jean-Daniel Cryans" <jdcryans@apache.org>
To: "Jean-Daniel Cryans" <jdcryans@apache.org>, jiraposter@review.hbase.org,
	dev@hbase.apache.org
Date: Wed, 08 Dec 2010 00:38:42 -0000
Message-ID: <20101208003842.23478.1790@ip-10-202-7-187.ec2.internal>


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1273/
-----------------------------------------------------------

Review request for hbase.


Summary
-------

Patch that parallelizes the splitting of the files using ThreadPoolExecutor=
 and Futures. The code is a bit ugly, but does the job really well as shown=
 during cluster testing (which also uncovered HBASE-3318).

One new behavior this patch adds is that it's now possible to rollback a sp=
lit because it took too long to split the files. I did some testing with a =
timeout of 5 secs on my cluster, even tho each machine did a few rollbacks =
the import went fine. The default is 30 seconds and isn't in hbase-default.=
xml as I don't think anyone would really want to change that.


This addresses bug HBASE-3308.
    http://issues.apache.org/jira/browse/HBASE-3308


Diffs
-----

  /branches/0.90/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTr=
ansaction.java 1043188 =


Diff: http://review.cloudera.org/r/1273/diff


Testing
-------


Thanks,

Jean-Daniel