hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Tovbin <matt...@tovbin.com>
Subject Increase number of reducers for bulk data load to empty HBase table
Date Tue, 18 Oct 2011 09:00:33 GMT
Hello, Guys,

I'm willing to bulk load data from hdfs folders into HBase, for this purpose
I used configureIncrementalLoad method from HFileOutputFormat that
configures the job, as follows:

org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(job,
myTable)

The problem is that destination table in HBase is empty, meaning it's only
hosted by one region server, so the resulted number of reducers is 1, which
makes the job to run almost forever.

How can I increase the number of reducers? Can the number of reducers be set
to more than a number of region servers?

Thanks in advance,
     Matthew Tovbin.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message