hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5847) Remove redundant code for fileOutputByteCounter in MapTask and ReduceTask
Date Thu, 05 Feb 2015 22:32:35 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14308140#comment-14308140
] 

Allen Wittenauer commented on MAPREDUCE-5847:
---------------------------------------------

This seems like such a low risk and, as it is today, aren't we actually reporting wrong information?
That's significantly worse!  (I know of one vendor that is actually mentions that they report
correct values for some metrics since we blow it so badly in lots of places...)

While I understand the concerns about moving from 2.x to 3.x, users should expect some degree
of pain when moving major versions. 

> Remove redundant code for fileOutputByteCounter in MapTask and ReduceTask 
> --------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5847
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5847
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv1, mrv2, task
>    Affects Versions: 2.4.0
>            Reporter: Gera Shegalov
>            Assignee: Gera Shegalov
>         Attachments: MAPREDUCE-5847.v01.patch, MAPREDUCE-5847.v02.patch
>
>
> Both MapTask and ReduceTask carry redundant code to update BYTES_WRITTEN counter. However,
{{Task.updateCounters}} uses file system stats for this. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message