hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "William Slacum (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2557) Counters don't reset state when readFields() called
Date Tue, 07 Jun 2011 01:38:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045236#comment-13045236
] 

William Slacum commented on MAPREDUCE-2557:
-------------------------------------------

Hey Todd,

1) No problem on that

2) I think in terms of running a MR job, there's no real effect unless the framework is re-using
the same Counters reference for different jobs, which could happen if JVM re-use is enabled.
I encountered this while writing an MR job that used Counters as an input value type. I had
written my own InputFormat that re-used a Counters reference and found I was getting odd values
when performing operations on Counter values.

> Counters don't reset state when readFields() called
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-2557
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2557
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: William Slacum
>            Priority: Trivial
>         Attachments: MAPREDUCE-2557.patch
>
>   Original Estimate: 0.5h
>  Remaining Estimate: 0.5h
>
> When calling readFields() on a Counters object, the internal state is not completely
reset. The IdentityHashMap<Enum<?>, Counter> cache retains all previous mappings,
even after the actual CounterGroups are changed. Using the same Counters pointer over and
over again results in the cache always keeping the mapping for the first call to getCounter(Enum<?>).
I've add a clear() call to the cache when readFields() is called and added a unit test to
verify that it works.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message