hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prabhu Joseph (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6975) Logging task counters
Date Fri, 06 Oct 2017 10:58:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16194451#comment-16194451
] 

Prabhu Joseph commented on MAPREDUCE-6975:
------------------------------------------

[~Naganarasimha]  Thanks for looking into this. Below is the output. Have changed it to a
new line in the .1 patch.

{code}
2017-10-05 16:00:40,535 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1506610341926_0006_m_000000_0'
done.
2017-10-05 16:00:40,542 INFO [main] org.apache.hadoop.mapred.Task: Final Counters for attempt_1506610341926_0006_m_000000_0:
File System Counters.FILE: Number of bytes read:0,File System Counters.FILE: Number of bytes
written:146949,File System Counters.FILE: Number of read operations:0,File System Counters.FILE:
Number of large read operations:0,File System Counters.FILE: Number of write operations:0,File
System Counters.HDFS: Number of bytes read:267,File System Counters.HDFS: Number of bytes
written:0,File System Counters.HDFS: Number of read operations:3,File System Counters.HDFS:
Number of large read operations:0,File System Counters.HDFS: Number of write operations:0,Map-Reduce
Framework.Map input records:3,Map-Reduce Framework.Map output records:6,Map-Reduce Framework.Map
output bytes:180,Map-Reduce Framework.Map output materialized bytes:133,Map-Reduce Framework.Input
split bytes:111,Map-Reduce Framework.Combine input records:6,Map-Reduce Framework.Combine
output records:4,Map-Reduce Framework.Spilled Records:4,Map-Reduce Framework.Failed Shuffles:0,Map-Reduce
Framework.Merged Map outputs:0,Map-Reduce Framework.GC time elapsed (ms):93,Map-Reduce Framework.CPU
time spent (ms):1400,Map-Reduce Framework.Physical memory (bytes) snapshot:1427480576,Map-Reduce
Framework.Virtual memory (bytes) snapshot:3663581184,Map-Reduce Framework.Total committed
heap usage (bytes):1402994688,File Input Format Counters .Bytes Read:156
{code}

> Logging task counters 
> ----------------------
>
>                 Key: MAPREDUCE-6975
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6975
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: task
>    Affects Versions: 2.7.3
>            Reporter: Prabhu Joseph
>            Assignee: Prabhu Joseph
>         Attachments: MAPREDUCE-6975.1.patch, MAPREDUCE-6975.patch
>
>
> Logging counters for each task at the end of it's syslog will make debug easier with
just application logs. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message