hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prabhu Joseph (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6975) Logging task counters
Date Tue, 10 Oct 2017 05:50:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16198203#comment-16198203
] 

Prabhu Joseph commented on MAPREDUCE-6975:
------------------------------------------

[~Naganarasimha] Will add the output as a comment in code. Have tried with Counters.toSting()
which outputs each counter in a separate line. I think having all counters in same line along
with task id will be easier to compare the counters of each task with awk and grep. We can
make it readable by not printing the group name for every counter the Group has which will
reduce the verbosity. Shall we refactor the Counters.makeCompactString similar to below way
what Tez uses.

https://github.com/apache/tez/blob/master/tez-api/src/main/java/org/apache/tez/common/counters/AbstractCounters.java#L314

{code}
2017-10-05 16:00:40,542 INFO [main] org.apache.hadoop.mapred.Task: Final Counters for attempt_1506610341926_0006_m_000000_0:
[[<groupname1> countername=value, countername1=value1,][<groupname2> countername=value,
countername1=value1,]]
{code}



> Logging task counters 
> ----------------------
>
>                 Key: MAPREDUCE-6975
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6975
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: task
>    Affects Versions: 2.7.3
>            Reporter: Prabhu Joseph
>            Assignee: Prabhu Joseph
>         Attachments: MAPREDUCE-6975.1.patch, MAPREDUCE-6975.patch
>
>
> Logging counters for each task at the end of it's syslog will make debug easier with
just application logs. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message