hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Torsten Curdt <tcu...@vafer.org>
Subject Re: limit of values in reduce phase?
Date Tue, 22 Jun 2010 07:04:49 GMT
Cool. Great :)

On Tue, Jun 22, 2010 at 07:47, Owen O'Malley <omalley@apache.org> wrote:
>
> On Jun 21, 2010, at 5:14 PM, Torsten Curdt wrote:
>
>> I was just wondering the other day:
>>
>> What if the the values for a key that get passed into the reducer do
>> not fit into memory?
>> After all a reducer should get all values per key from the whole job.
>> Is the iterator disk backed?
>
> There is no assumption that all of the values fit into memory. The iterator
> is really the
> result of a merge sort from disk and/or memory.
>
> -- Owen=
>

Mime
View raw message