hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5585) FileSystem statistic counters are too high when JVM reuse is enabled.
Date Mon, 30 Mar 2009 07:58:50 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Owen O'Malley updated HADOOP-5585:
----------------------------------

    Attachment: h5585.patch

This patch:
  1. Adds a reset method for statistics, so that when the framework switches to a new task,
it can clean the counters.
  2. Removes the map from scheme to statistics, since it was only used to iterate through.
  3. Removes a race condition where the map was passed back out of the synchronized method,
which could cause ConcurrentModificationException.

> FileSystem statistic counters are too high when JVM reuse is enabled.
> ---------------------------------------------------------------------
>
>                 Key: HADOOP-5585
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5585
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>            Priority: Blocker
>             Fix For: 0.20.0
>
>         Attachments: h5585.patch
>
>
> When JVM reuse is enabled, the FileSystem.Statistics are not cleared between tasks. That
means that the second task gets credit for its own reads and writes as well as the first.
The third gets credit for all 3 tasks reads and writes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message