hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Tovbin <matt...@tovbin.com>
Subject Re: Increase number of reducers for bulk data load to empty HBase table
Date Sun, 23 Oct 2011 10:27:55 GMT
Worked like a charm. Thanks! ;)

Best regards,
   Matthew Tovbin =)



On Tue, Oct 18, 2011 at 19:30, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> (putting dev@ in bcc, please don't cross-post)
>
> You need to pre-split that table, see
> http://hbase.apache.org/book.html#precreate.regions
>
> J-D
>
> On Tue, Oct 18, 2011 at 2:00 AM, Matthew Tovbin <matthew@tovbin.com>
> wrote:
>
> > Hello, Guys,
> >
> > I'm willing to bulk load data from hdfs folders into HBase, for this
> > purpose
> > I used configureIncrementalLoad method from HFileOutputFormat that
> > configures the job, as follows:
> >
> >
> >
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(job,
> > myTable)
> >
> > The problem is that destination table in HBase is empty, meaning it's
> only
> > hosted by one region server, so the resulted number of reducers is 1,
> which
> > makes the job to run almost forever.
> >
> > How can I increase the number of reducers? Can the number of reducers be
> > set
> > to more than a number of region servers?
> >
> > Thanks in advance,
> >      Matthew Tovbin.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message