hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-6958) Shuffle audit logger should log size of shuffle transfer
Date Thu, 14 Sep 2017 21:54:00 GMT
Jason Lowe created MAPREDUCE-6958:

             Summary: Shuffle audit logger should log size of shuffle transfer
                 Key: MAPREDUCE-6958
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6958
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
            Reporter: Jason Lowe
            Assignee: Jason Lowe
            Priority: Minor

The shuffle audit logger currently logs the job ID and reducer ID but nothing about the size
of the requested transfer.  It calculates this as part of the HTTP response headers, so it
would be trivial to log the response size.  This would be very valuable for debugging network
traffic storms from the shuffle handler.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org

View raw message