Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Message-ID: <1335618978.418071269298047382.JavaMail.jira@brutus.apache.org>
Date: Mon, 22 Mar 2010 22:47:27 +0000 (UTC)
From: "Chris Douglas (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Subject: [jira] Updated: (MAPREDUCE-1304) Add counters for task time spent
 in GC
In-Reply-To: <1562893346.1260999858064.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/MAPREDUCE-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Douglas updated MAPREDUCE-1304:
-------------------------------------

    Status: Open  (was: Patch Available)

Great idea. A few comments on the impl:
* This has no effect:
{noformat}
+    job.getConfiguration().set("io.sort.record.pct", "0.50");
{noformat}
* Moving the update of the GC counter to {{Task::updateCounters}} makes sense. To capture any {{FileSystem}} activity in the committer, the call to {{updateCounters}} in {{Task::done}} should be after the commit, anyway. This is a bug in the current code.
* Adding {{java.lang.management}} components shouldn't be widening the {{TaskReporter}} API and adding fields. There should be an {{\*Updater}} object tracking the particular statistic being tracked (as in the {{FileSystem}} counters) rather than letting the {{TaskReporter}} keep this state.
* Updating the GC counters must be synchronized or it risks reporting incorrect results (moving the update to {{Task::updateCounters}} would be sufficient).

> Add counters for task time spent in GC
> --------------------------------------
>
>                 Key: MAPREDUCE-1304
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1304
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: task
>            Reporter: Todd Lipcon
>            Assignee: Aaron Kimball
>         Attachments: MAPREDUCE-1304.2.patch, MAPREDUCE-1304.patch
>
>
> It's easy to grab the number of millis spent in GC (see JvmMetrics for example). Exposing these as task counters would be handy - occasionally I've seen user jobs where long GC pauses cause big "unexplainable" performance problems, and a large counter would make it obvious to the user what's going on.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.