Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of saint.ack@gmail.com designates
 209.85.216.176 as permitted sender)
MIME-Version: 1.0
Sender: saint.ack@gmail.com
In-Reply-To: 
 <CANvfXVzPFKRYEQyqFkDDWndUKgcZMCqSegfeL0+JxZK4JxaLLA@mail.gmail.com>
References: 
 <CANvfXVw_7Vnv8kPyKnGhBj+vVe2eX-+3q+xV5uiYgetxd_Y+_Q@mail.gmail.com>
	<CANvfXVzPFKRYEQyqFkDDWndUKgcZMCqSegfeL0+JxZK4JxaLLA@mail.gmail.com>
Date: Wed, 14 Sep 2011 22:14:22 -0700
Message-ID: 
 <CADcMMgGCZmWTToYf7pxbDwf1eYGSrZmKHMun2ZOjHqV2bk__3w@mail.gmail.com>
Subject: Re: HBase ImportTSV
From: Stack <stack@duboce.net>
To: user@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1

Do you know your keyspace roughly?  Try creating a pre-split table
with as many regions as you want reducers.
St.Ack

On Wed, Sep 14, 2011 at 8:25 PM, rajesh balamohan
<rbalamohan2k@gmail.com> wrote:
> ImportTSV internally uses HFileOutputFormat.configureIncrementalLoad(job,
> table);
>
> However, for newly created tables there would not be any keys available.
> Hence, it launches 1 reducer by default.
>
> Is there a way to increase the number of reducers for high volume imports
> like 500+ GB.
>
> ~Rajesh.B
>
> On Thu, Sep 15, 2011 at 8:51 AM, rajesh balamohan <rbalamohan2k@gmail.com>wrote:
>
>> Hi All,
>>
>> ImportTSV is a great tool for bulk loading the data into HBASE.
>>
>> I have close to 500+GB of raw data which I would like to import into a
>> newly created HTABLE. If I go ahead with ImportTSV, it creates only one
>> reducer which is a bottleneck in terms of sorting and shuffling.
>>
>> Are there any other way, I can increase the number of reducers while doing
>> bulk loads for new table?.
>>
>> ~Rajesh.B
>>
>