Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 15 Sep 2017 15:20:00 +0000 (UTC)
From: "Rushabh S Shah (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <JIRA.13102398.1505425986000.123759.1505488800167@Atlassian.JIRA>
In-Reply-To: <JIRA.13102398.1505425986000@Atlassian.JIRA>
References: <JIRA.13102398.1505425986000@Atlassian.JIRA> <JIRA.13102398.1505425986447@jira-lw-us.apache.org>
Subject: [jira] [Comment Edited] (MAPREDUCE-6958) Shuffle audit logger
 should log size of shuffle transfer
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 15 Sep 2017 15:20:04 -0000


    [ https://issues.apache.org/jira/browse/MAPREDUCE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168013#comment-16168013 ] 

Rushabh S Shah edited comment on MAPREDUCE-6958 at 9/15/17 3:19 PM:
--------------------------------------------------------------------

Overall the patch looks good.
Just one very minor nit.
{noformat}
        for (String mapId : mapIds) {
           sb.append(" ");
          sb.append(mapId);
        }
{noformat}
Instead of going through {{mapId}}, we can just print {{mapIds}}. It will be just less code.


was (Author: shahrs87):
Overall the patch looks good.
Just one very minor nit.
{quote}
+        for (String mapId : mapIds) {
+          sb.append(" ");
+          sb.append(mapId);
+        }
{quote}
Instead of going through {{mapId}}, we can just print {{mapIds}}. It will be just less code.

> Shuffle audit logger should log size of shuffle transfer
> --------------------------------------------------------
>
>                 Key: MAPREDUCE-6958
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6958
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>            Priority: Minor
>         Attachments: MAPREDUCE-6958.001.patch
>
>
> The shuffle audit logger currently logs the job ID and reducer ID but nothing about the size of the requested transfer.  It calculates this as part of the HTTP response headers, so it would be trivial to log the response size.  This would be very valuable for debugging network traffic storms from the shuffle handler.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org