Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of klzhao@gmail.com designates
 66.249.82.237 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:content-transfer-encoding:content-disposition
         :references;
        b=ZNbrJH0RNj3QtsgxZfArpS25usb/yhr1S+WIXjh6ikPXjuRAviOPMtKk3g3iPI7XHH
         Qhz8O18/0wORDMwas6bmluwP3v2U3MSbkTbVPPil9Xrl8WcujHsPQQEdoUeN4PVM7L8F
         sniJh7nemNyZ9IbRcuNP3Qf6AfVrUSslFSLI8=
Message-ID: <ec25226e0808221616j16e8a362q9b712be3f1971cb0@mail.gmail.com>
Date: Fri, 22 Aug 2008 16:16:27 -0700
From: Kevin <klzhao@gmail.com>
To: core-user@hadoop.apache.org
Subject: Re: Question about distributed sort
In-Reply-To: <b297a0a70808221606s7a687151u1e76a411404d9443@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <b297a0a70808221600o529b2f3fn1177bd3246d53d82@mail.gmail.com>
	 <ec25226e0808221602s4831546axb4786f2b2d95ee0d@mail.gmail.com>
	 <b297a0a70808221606s7a687151u1e76a411404d9443@mail.gmail.com>

For the same key, reducer is called only once.
-Kevin


On Fri, Aug 22, 2008 at 4:06 PM, Alex Holmes <grep.alex@gmail.com> wrote:
> If this is the case, can the same reducer be invoked multiple times
> with the same key?  And if so, would this imply that the key could
> appear on multiple lines of the reducer output file?
>
> Thanks,
> Alex
>
> On Fri, Aug 22, 2008 at 7:02 PM, Kevin <klzhao@gmail.com> wrote:
>> IIRC, the same key will always be sent to the same reducer.
>> -Kevin
>>
>>
>>
>> On Fri, Aug 22, 2008 at 4:00 PM, Alex Holmes <grep.alex@gmail.com> wrote:
>>> Hi,
>>>
>>> For a given input key, K, in a reduce task, does Hadoop guarantee that
>>> all mapper-emitted values for key K are available in the iterator?  Is
>>> it possible that multiple reduce tasks can receive the same key?
>>>
>>> Or to phrase the question in another way, for a single map-reduce job,
>>> where you have multiple mapper and multiple reducer tasks, is there a
>>> possibility that the same key appears in multiple reduce output files
>>> (assuming the reducer only emits a single output K,V pair, where the
>>> output K is identical to the input K)?
>>>
>>> Any assistance would be greatly appreciated.
>>>
>>> Thanks,
>>> Alex
>>>
>>
>