hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <omal...@apache.org>
Subject Re: limit of values in reduce phase?
Date Tue, 22 Jun 2010 05:47:38 GMT

On Jun 21, 2010, at 5:14 PM, Torsten Curdt wrote:

> I was just wondering the other day:
>
> What if the the values for a key that get passed into the reducer do
> not fit into memory?
> After all a reducer should get all values per key from the whole job.
> Is the iterator disk backed?

There is no assumption that all of the values fit into memory. The  
iterator is really the
result of a merge sort from disk and/or memory.

-- Owen=

Mime
View raw message