Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: neutral (athena.apache.org: 75.101.130.251 is neither permitted
 nor denied by domain of stack@duboce.net)
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Subject: Re: Review Request: HBASE-2468: Improvements to prewarm META cache on
 clients.
From: stack@duboce.net
To: "Todd Lipcon" <todd@cloudera.com>, stack@duboce.net
Date: Sun, 13 Jun 2010 17:19:10 -0000
Message-ID: <20100613171910.21500.10225@ip-10-250-218-255.ec2.internal>
Cc: "Mingjie Lai" <mjlai09@gmail.com>, jiraposter@review.hbase.org,
	dev@hbase.apache.org
In-Reply-To: <20100613074951.21500.34231@ip-10-250-218-255.ec2.internal>
References: <20100613074951.21500.34231@ip-10-250-218-255.ec2.internal>


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/98/#review210
-----------------------------------------------------------

Ship it!


+1

I did a quick skim.  All looks good.  This is a weird one in that to prewar=
m we are going to do a getClosestOrBefore (which we'll be doing anyway) and=
 then we'll open scanner, and scan next 10 rows... then close scanner; i.e.=
 extra rpcs inline w/ first lookup against tale.  This latter will actually=
 slow down first lookup some but we can make it faster by making open scann=
er return results so we don't have to then go call next (already an issue t=
o do this) and also, making it so we scan forward always by changing keys i=
n .META. such that region names are keyed by endkey rather than startkey...=
 also another issue to do this.

- stack


On 2010-06-13 00:49:50, Mingjie Lai wrote:
> =

> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.hbase.org/r/98/
> -----------------------------------------------------------
> =

> (Updated 2010-06-13 00:49:50)
> =

> =

> Review request for hbase, Todd Lipcon and stack.
> =

> =

> Summary
> -------
> =

> HBASE-2468: Improvements to prewarm META cache on clients.
> =

> Changes:
> 1. Add new HTable methods which support region info de/serialation, and r=
egion cache prewarm: =

> - void serializeRegionInfo(): clients could perform a large scan for all =
the meta for the table, serialize the meta to a file. MR job can ship a cop=
y of the meta for the table in the DistributedCache
> - Map<HRegionInfo, HServerAddress> deserializeRegionInfo(): MR job can de=
serialize the region info from the DistributedCache =

> - prewarmRegionCache(Map<HRegionInfo, HServerAddress> regionMap): MR job =
can prewarm local region cache by the deserialized region info.
> =

> 2. For each client, each region cache read-miss could trigger read-ahead =
some number of rows in META. This option could be turned on and off for one=
 particular table. =

> =

> =

> This addresses bug HBASE-2468.
>     http://issues.apache.org/jira/browse/HBASE-2468
> =

> =

> Diffs
> -----
> =

>   src/main/java/org/apache/hadoop/hbase/client/HConnection.java 853164d =

>   src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 03=
cbf8d =

>   src/main/java/org/apache/hadoop/hbase/client/HTable.java 8e47f2d =

>   src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java f11ada7 =

>   src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java 95=
e494a =

> =

> Diff: http://review.hbase.org/r/98/diff
> =

> =

> Testing
> -------
> =

> Unit tests passed locally for me. =

> =

> =

> Thanks,
> =

> Mingjie
> =

>