hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: TableInputFormat improvement to handle lots of small regions
Date Fri, 01 Jul 2011 05:58:00 GMT
On Thu, Jun 30, 2011 at 8:38 AM, Ophir Cohen <ophchu@gmail.com> wrote:
> Actually I thought of opposite version:
> If I have a spare map slots why not configure it to run more than one mapper
> on region?
> The question then is how to 'skip' the mappers to the needed places inside
> the regions.

Well, the current splitter passed mappers Scans where the start/end
rows are the region boundaries (at the time at which the splitter

To do your case,  in the splitter, you'd just give out multiple splits
per region.  To cut up the region key-space, you might use the
Bytes.split code.  It does coarse BigNumber math dividing the key
space.  See here:


View raw message