Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=cc:message-id:from:to:in-reply-to:content-type:
	content-transfer-encoding:mime-version:subject:date:references:x-mailer;
	b=RmYWh/a2Ni9++wvKR5mS0i9AAxGBuhDjlfeWLsebxK+8Y3HKJ6n3AU5ZVi4x3sRS
Cc: common-dev@hadoop.apache.org
Message-Id: <63609015-F12D-4054-BF96-3D8477036747@yahoo-inc.com>
From: Arun C Murthy <acm@yahoo-inc.com>
To: mapreduce-dev@hadoop.apache.org
In-Reply-To: <1c9a863e0908020502n7d810962rbc561437388fce84@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed; delsp=yes
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: why reduce task can be scheduled before map tasks are 100%
 completed?
Date: Sun, 2 Aug 2009 20:57:05 -0700
References: <1c9a863e0908020502n7d810962rbc561437388fce84@mail.gmail.com>

That check ensures sufficient #maps are completed before any of the =20
reduces for the job are started.

The reduces 'shuffle' map outputs from completed maps, but don't get =20
into the 'reduce' phase until all map-outputs are copied over.

Arun

PS: Moving this to mapreduce-dev@

On Aug 2, 2009, at 5:02 AM, =E6=88=91=E7=9A=84Gmail=E9=82=AE=E7=AE=B1 =
wrote:

> Hi, everyone.
> In class org.apache.hadoop.mapred.JobInProgress, there is a public =20
> method:
> scheduleReduces(), it will return true if "finishedMapTasks >=3D
> completedMapsForReduceSlowstart"
> and then the scheduler can schedule a new reduce task for a given
> taskTracker.
>
> but as I konw, reduce can not be started unitl map is 100% =20
> completed. Does
> anyone can explain it? thanks a lot.