hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manish Verma <manish.lifepa...@gmail.com>
Subject Re: incrCounter doesn't return a value
Date Sun, 04 Aug 2013 18:06:20 GMT
Jens/Ashish,
Thanks for your replies. I missed the point that these counters are
maintained by their respective tasks and periodically sent to the
tasktracker and then to the jobtracker for global aggregation.

Thanks
Manish


On Sun, Aug 4, 2013 at 10:43 AM, Ashish Umrani <ashish.umrani@gmail.com>wrote:

> Manish,
>
> I am not sure if the counter would provide a globally unique id.  To the
> best of my knowledge, Counters are mapper specific.  So even if one could
> see the application working on one mapper, at the end, when deployed on
> production, duplicate ids would cause a problem.  So, unless you are
> looking for uniqueness on a single mapper, I would suggest you to look for
> some other solution, perhaps like the one suggested by Jens.
>
> Regards
> ashish
>
>
> On Sun, Aug 4, 2013 at 10:36 AM, Jens Scheidtmann <
> jens.scheidtmann@gmail.com> wrote:
>
>> Dear Manish,
>>
>> Use some combination of Job Id and/or IP address and other attributes to
>> make your own unique ids. incrCounter had to synchronize globally across
>> your cluster, if it were to provide incremental IDs.
>>
>> Best regards,
>>
>> Jens
>>
>> Am Donnerstag, 1. August 2013 schrieb Manish Verma :
>>
>>> Hi,
>>>
>>> I wanted to use the incrCounter API to generate auto increment ids but
>>> the problem is that it doesn't return the incremented value. Does anyone
>>> know why this API does not return the incremented value? And whether it
>>> would be possible to change it to return the incremented value?
>>>
>>> Thanks
>>> Manish V
>>>
>>
>

Mime
View raw message