hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From st...@duboce.net
Subject Re: Review Request: HBASE-2468: Improvements to prewarm META cache on clients.
Date Sun, 13 Jun 2010 17:19:10 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/98/#review210
-----------------------------------------------------------

Ship it!


+1

I did a quick skim.  All looks good.  This is a weird one in that to prewarm we are going
to do a getClosestOrBefore (which we'll be doing anyway) and then we'll open scanner, and
scan next 10 rows... then close scanner; i.e. extra rpcs inline w/ first lookup against tale.
 This latter will actually slow down first lookup some but we can make it faster by making
open scanner return results so we don't have to then go call next (already an issue to do
this) and also, making it so we scan forward always by changing keys in .META. such that region
names are keyed by endkey rather than startkey... also another issue to do this.

- stack


On 2010-06-13 00:49:50, Mingjie Lai wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.hbase.org/r/98/
> -----------------------------------------------------------
> 
> (Updated 2010-06-13 00:49:50)
> 
> 
> Review request for hbase, Todd Lipcon and stack.
> 
> 
> Summary
> -------
> 
> HBASE-2468: Improvements to prewarm META cache on clients.
> 
> Changes:
> 1. Add new HTable methods which support region info de/serialation, and region cache
prewarm: 
> - void serializeRegionInfo(): clients could perform a large scan for all the meta for
the table, serialize the meta to a file. MR job can ship a copy of the meta for the table
in the DistributedCache
> - Map<HRegionInfo, HServerAddress> deserializeRegionInfo(): MR job can deserialize
the region info from the DistributedCache 
> - prewarmRegionCache(Map<HRegionInfo, HServerAddress> regionMap): MR job can prewarm
local region cache by the deserialized region info.
> 
> 2. For each client, each region cache read-miss could trigger read-ahead some number
of rows in META. This option could be turned on and off for one particular table. 
> 
> 
> This addresses bug HBASE-2468.
>     http://issues.apache.org/jira/browse/HBASE-2468
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/hadoop/hbase/client/HConnection.java 853164d 
>   src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java 03cbf8d 
>   src/main/java/org/apache/hadoop/hbase/client/HTable.java 8e47f2d 
>   src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java f11ada7 
>   src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java 95e494a 
> 
> Diff: http://review.hbase.org/r/98/diff
> 
> 
> Testing
> -------
> 
> Unit tests passed locally for me. 
> 
> 
> Thanks,
> 
> Mingjie
> 
>


Mime
View raw message