hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Pearson" <sa...@pearsonwholesale.com>
Subject Re: how many map in a map task?
Date Mon, 10 Nov 2008 15:14:33 GMT
There is no way I know of to get the number of inputs records from the input 
unless you only write 100 to the input file.
Might find better help on this from the hadoop user group they know the MR 
framework better then me.

Billy



"ma qiang" <maqiang1984@gmail.com> wrote in 
message news:2af70f430811100049o5f599ec4nc2672e932cc919b7@mail.gmail.com...
> Thanks very much.
> However, I want to use the number of the map in my program. for
> example.  There are 100 map in a map task, and I want to collect all
> the values, and analyze these values then output the last result in
> the last map.
> generally, there is no ouput in the first 99 map, only output exists
> in the last map.
> So I should know how many map in a map task.
>
>
>
> On Mon, Nov 10, 2008 at 4:33 PM, Billy Pearson
> <sales@pearsonwholesale.com> wrote:
>> If you are using TableMap to read the data back each mapper will map over
>> all the rows in a single region.
>>
>> You can use counters to count the records so we can get a total records.
>>
>> but there is not set number of maps a mapper will process it depends on 
>> the
>> input.
>>
>> Billy
>>
>>
>>
>> "ma qiang" <maqiang1984@gmail.com> wrote in 
>> message
>> news:2af70f430811100011n5323ff03h3417aac50d5f0cc6@mail.gmail.com...
>>>
>>> hi all,
>>>    I hava a data set stored in hbase, and I run a mapreduce program
>>> to analyze. Now I want to know how many map in a map task?  Who can
>>> tell me?
>>>    Thanks a lot.
>>>
>>> yours qiang.
>>>
>>
>>
>>
> 



Mime
View raw message