Return-Path: Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: (qmail 8497 invoked from network); 22 Jun 2010 07:05:22 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 22 Jun 2010 07:05:22 -0000 Received: (qmail 85279 invoked by uid 500); 22 Jun 2010 07:05:22 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 84763 invoked by uid 500); 22 Jun 2010 07:05:18 -0000 Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-user@hadoop.apache.org Delivered-To: mailing list mapreduce-user@hadoop.apache.org Received: (qmail 84752 invoked by uid 99); 22 Jun 2010 07:05:17 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Jun 2010 07:05:17 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=AWL,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [74.125.78.26] (HELO ey-out-2122.google.com) (74.125.78.26) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Jun 2010 07:05:11 +0000 Received: by ey-out-2122.google.com with SMTP id 22so385986eye.23 for ; Tue, 22 Jun 2010 00:04:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.213.10.195 with SMTP id q3mr1272095ebq.59.1277190289361; Tue, 22 Jun 2010 00:04:49 -0700 (PDT) Received: by 10.213.32.81 with HTTP; Tue, 22 Jun 2010 00:04:49 -0700 (PDT) In-Reply-To: References: Date: Tue, 22 Jun 2010 09:04:49 +0200 Message-ID: Subject: Re: limit of values in reduce phase? From: Torsten Curdt To: mapreduce-user@hadoop.apache.org Content-Type: text/plain; charset=ISO-8859-1 Cool. Great :) On Tue, Jun 22, 2010 at 07:47, Owen O'Malley wrote: > > On Jun 21, 2010, at 5:14 PM, Torsten Curdt wrote: > >> I was just wondering the other day: >> >> What if the the values for a key that get passed into the reducer do >> not fit into memory? >> After all a reducer should get all values per key from the whole job. >> Is the iterator disk backed? > > There is no assumption that all of the values fit into memory. The iterator > is really the > result of a merge sort from disk and/or memory. > > -- Owen= >