hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karthik Ranganathan <kranganat...@fb.com>
Subject Re: Increase number of reducers for bulk data load to empty HBase table
Date Wed, 19 Oct 2011 15:41:50 GMT
Hey Matthew,

The only way to increase the number of reducers is to have more regions -
each reducer produces an output per region, so the number of reducers ==
number of regions.

Thanks
Karthik


On 10/18/11 2:00 AM, "Matthew Tovbin" <matthew@tovbin.com> wrote:

>Hello, Guys,
>
>I'm willing to bulk load data from hdfs folders into HBase, for this
>purpose
>I used configureIncrementalLoad method from HFileOutputFormat that
>configures the job, as follows:
>
>org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLo
>ad(job,
>myTable)
>
>The problem is that destination table in HBase is empty, meaning it's only
>hosted by one region server, so the resulted number of reducers is 1,
>which
>makes the job to run almost forever.
>
>How can I increase the number of reducers? Can the number of reducers be
>set
>to more than a number of region servers?
>
>Thanks in advance,
>     Matthew Tovbin.


Mime
View raw message