hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin <klz...@gmail.com>
Subject Re: Question about distributed sort
Date Fri, 22 Aug 2008 23:16:27 GMT
For the same key, reducer is called only once.
-Kevin



On Fri, Aug 22, 2008 at 4:06 PM, Alex Holmes <grep.alex@gmail.com> wrote:
> If this is the case, can the same reducer be invoked multiple times
> with the same key?  And if so, would this imply that the key could
> appear on multiple lines of the reducer output file?
>
> Thanks,
> Alex
>
> On Fri, Aug 22, 2008 at 7:02 PM, Kevin <klzhao@gmail.com> wrote:
>> IIRC, the same key will always be sent to the same reducer.
>> -Kevin
>>
>>
>>
>> On Fri, Aug 22, 2008 at 4:00 PM, Alex Holmes <grep.alex@gmail.com> wrote:
>>> Hi,
>>>
>>> For a given input key, K, in a reduce task, does Hadoop guarantee that
>>> all mapper-emitted values for key K are available in the iterator?  Is
>>> it possible that multiple reduce tasks can receive the same key?
>>>
>>> Or to phrase the question in another way, for a single map-reduce job,
>>> where you have multiple mapper and multiple reducer tasks, is there a
>>> possibility that the same key appears in multiple reduce output files
>>> (assuming the reducer only emits a single output K,V pair, where the
>>> output K is identical to the input K)?
>>>
>>> Any assistance would be greatly appreciated.
>>>
>>> Thanks,
>>> Alex
>>>
>>
>

Mime
View raw message