hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: TableInputFormat improvement to handle lots of small regions
Date Fri, 01 Jul 2011 05:58:00 GMT
On Thu, Jun 30, 2011 at 8:38 AM, Ophir Cohen <ophchu@gmail.com> wrote:
> Actually I thought of opposite version:
> If I have a spare map slots why not configure it to run more than one mapper
> on region?
> The question then is how to 'skip' the mappers to the needed places inside
> the regions.

Well, the current splitter passed mappers Scans where the start/end
rows are the region boundaries (at the time at which the splitter
ran).

To do your case,  in the splitter, you'd just give out multiple splits
per region.  To cut up the region key-space, you might use the
Bytes.split code.  It does coarse BigNumber math dividing the key
space.  See here:
http://hbase.apache.org/xref/org/apache/hadoop/hbase/util/Bytes.html#1034

St.Ack

Mime
View raw message