Return-Path: 
 <user-return-11005-apmail-hadoop-mapreduce-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org
Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 0C3251035C
	for <apmail-hadoop-mapreduce-user-archive@minotaur.apache.org>;
 Wed, 25 Sep 2013 01:58:42 +0000 (UTC)
Received: (qmail 72085 invoked by uid 500); 25 Sep 2013 01:58:37 -0000
Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org
Received: (qmail 71937 invoked by uid 500); 25 Sep 2013 01:58:37 -0000
Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:user@hadoop.apache.org>
List-Id: <user.hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Delivered-To: mailing list user@hadoop.apache.org
Received: (qmail 71930 invoked by uid 99); 25 Sep 2013 01:58:37 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 25 Sep 2013 01:58:37 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of efegencer@gmail.com
 designates 209.85.219.47 as permitted sender)
Received: from [209.85.219.47] (HELO mail-oa0-f47.google.com) (209.85.219.47)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 25 Sep 2013 01:58:32 +0000
Received: by mail-oa0-f47.google.com with SMTP id i1so272764oag.6
        for <user@hadoop.apache.org>; Tue, 24 Sep 2013 18:58:11 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type;
        bh=5FP9xFBRlk9Rrz2nFU86BEDDgycH2LHUnbsPvEZ5lmI=;
        b=z/bzqZWFzd36N0giAFEPq3k9/pgUu7YBI0DkZUe7GNH+XL56qQ2PTX7EMp9Qsy1Ow/
         h3Hg+kNTl5CAG0uSKOckEb5RmBasjtOWt/qe9T2quHrc9tyeq+6ji1L7ClDs2z2p+f5r
         0LkvqRnJ4p5hhbWC4v3zMzypgyz9q3hMzDmvqCDePj9kEHt0QPzbBpEYC+aAQ+ecIOvM
         Ec6crXzgbmMODQnzwFd3LieqwvKRQ5DC7owp4y1EjIMJjjAtKtgs7o6kZA3nYif4f166
         afNLACn/8nj+Vz6zHaos6+gdBjfEee+tTGLcwOae8PlpKndWH2OiXp/PwBvTx5w00PE3
         DPJA==
X-Received: by 10.182.104.130 with SMTP id ge2mr28003407obb.6.1380074291553;
 Tue, 24 Sep 2013 18:58:11 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.182.85.34 with HTTP; Tue, 24 Sep 2013 18:57:51 -0700 (PDT)
In-Reply-To: 
 <CADz_ftb_FFnHX71PK9SDhZAXPVC7gY3dbuMHRKbOz24EL0Ez4g@mail.gmail.com>
References: 
 <CADz_ftb_FFnHX71PK9SDhZAXPVC7gY3dbuMHRKbOz24EL0Ez4g@mail.gmail.com>
From: Efe Gencer <efegencer@gmail.com>
Date: Tue, 24 Sep 2013 21:57:51 -0400
Message-ID: 
 <CADz_ftatwJf__Hm6Px8iBaETbJX0Q9OMYJNCZtVDiToT9dG+Cg@mail.gmail.com>
Subject: Re: Which Subphases Do Times on JobHistory Web UI Cover
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=089e0112cf10e96c8a04e72b9515
X-Virus-Checked: Checked by ClamAV on apache.org

--089e0112cf10e96c8a04e72b9515
Content-Type: text/plain; charset=ISO-8859-1

*By the way this question is about Apache Hadoop Release 2.1.0-beta.

Thanks,


2013/9/24 Efe Gencer <efegencer@gmail.com>

> Hi All,
>
> In JobHistory Web UI under Job > "Map Tasks" I see something as follows:
> ...
> Started: <start time>
> Finished: <finish time>
> Elapsed: 12 mins, 5sec
> Diagnostics:
> *Average Map Time*: 1 mins, 40 sec
> Average Reduce Time: 12 sec
> Average Shuffle Time: 10 mins, 8 sec
> Average Merge Time: 1 sec
> ...
>
> 1) I wonder which sub-map phases "Map Time" contains (e.g. map function,
> sort, spill, merge, read and transfer corresponding filesplit from HDFS)
> 2) Does Reduce time covers Shuffle Time? What else does it cover? (write
> to hdfs, etc)
> 3) Is Average Merge time related with map or reduce? (since they both have
> merge phases)
>
> Best,
> Efe
>

--089e0112cf10e96c8a04e72b9515
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>*By the way this question is about Apache Hadoop Rele=
ase 2.1.0-beta.<br><br></div>Thanks,<br><div class=3D"gmail_extra"><br><br>=
<div class=3D"gmail_quote">2013/9/24 Efe Gencer <span dir=3D"ltr">&lt;<a hr=
ef=3D"mailto:efegencer@gmail.com" target=3D"_blank">efegencer@gmail.com</a>=
&gt;</span><br>


<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hi All,<div><br></div><div>=
In JobHistory Web UI under Job &gt; &quot;Map Tasks&quot; I see something a=
s follows:</div>


<div>...</div><div>Started: &lt;start time&gt;</div><div>Finished: &lt;fini=
sh time&gt;</div>
<div>Elapsed: 12 mins, 5sec</div><div>Diagnostics:</div><div><b>Average Map=
 Time</b>: 1 mins, 40 sec</div><div>Average Reduce Time: 12 sec<br></div><d=
iv>Average Shuffle Time: 10 mins, 8 sec<br></div><div>Average Merge Time: 1=
 sec<br>



</div><div>...</div><div><br></div><div>1) I wonder which sub-map phases &q=
uot;Map Time&quot;=A0contains (e.g. map function, sort, spill, merge, read =
and transfer corresponding filesplit from HDFS)</div><div>2) Does Reduce ti=
me covers Shuffle Time? What else does it cover? (write to hdfs, etc)</div>



<div>3) Is Average Merge time related with map or reduce? (since they both =
have merge phases)</div><div><br></div><div>Best,</div><div>Efe</div></div>
</blockquote></div><br></div></div>

--089e0112cf10e96c8a04e72b9515--