hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Increase number of reducers for bulk data load to empty HBase table
Date Tue, 18 Oct 2011 17:30:46 GMT
(putting dev@ in bcc, please don't cross-post)

You need to pre-split that table, see
http://hbase.apache.org/book.html#precreate.regions

J-D

On Tue, Oct 18, 2011 at 2:00 AM, Matthew Tovbin <matthew@tovbin.com> wrote:

> Hello, Guys,
>
> I'm willing to bulk load data from hdfs folders into HBase, for this
> purpose
> I used configureIncrementalLoad method from HFileOutputFormat that
> configures the job, as follows:
>
>
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(job,
> myTable)
>
> The problem is that destination table in HBase is empty, meaning it's only
> hosted by one region server, so the resulted number of reducers is 1, which
> makes the job to run almost forever.
>
> How can I increase the number of reducers? Can the number of reducers be
> set
> to more than a number of region servers?
>
> Thanks in advance,
>      Matthew Tovbin.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message