Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of imperros@gmail.com designates
 209.85.214.41 as permitted sender)
Message-ID: <50214A23.8040101@gmail.com>
Date: Tue, 07 Aug 2012 20:02:27 +0300
From: Ioakim Perros <imperros@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux i686;
 rv:14.0) Gecko/20120714 Thunderbird/14.0
MIME-Version: 1.0
To: user@hbase.apache.org
Subject: Re: Bulk load - #Reducers different from #Regions
References: <5021447B.5060000@gmail.com>
 <CA+m5d=wuMx8d37_1Osq9FJ6fAB2_qxjNZXhEe4n6be=x4y5dzA@mail.gmail.com>
In-Reply-To: 
 <CA+m5d=wuMx8d37_1Osq9FJ6fAB2_qxjNZXhEe4n6be=x4y5dzA@mail.gmail.com>
Content-Type: multipart/alternative;
 boundary="------------070405090005050608020807"

--------------070405090005050608020807
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Excuse me for not well-defining.

I am bulk updating my hbase table through code, using 
configureIncrementalLoad function of HFileOutputFormat. At the 
respective documentation, I read that this function " Sets the number of 
reduce tasks to match the current number of regions" ,but I was 
wondering if I could explicitly avoid it, perhaps by another way of bulk 
importing data.

PS: I try to insist on bulk importing, because I have understood (I hope 
that this is correct), that it is much more efficient than going with 
the traditional Hbase API. And as I require my job to be of iterative 
nature, this way hopefully would end up giving a good boost-up, as 
opposed to the Hbase API.

Thank you for responding.


On 08/07/2012 07:53 PM, Subir S wrote:
> Bulk load using
> ImportTsv with pre-splitted regions for target table?
>
> Do u mean to set number of reducers that ImportTsv must use?
>
> On 8/7/12, Ioakim Perros <imperros@gmail.com> wrote:
>> HI,
>>
>> I am bulk importing (updating) data iteratively and I would like to be
>> able to set the number of reducers at a M/R task, to be different from
>> the number of regions of the table to which I am updating data.
>>
>> I tried it through job.setNumReduceTasks(#reducers), but the job ignored
>> it.
>>
>> Is there a way to avoid an intermediary job and to set the number of
>> reducers explicitly ?
>> I would be grateful if anyone could shed a light to this.
>>
>> Thanks and regards,
>> Ioakim
>>
>>


--------------070405090005050608020807--