Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Message-ID: <2749576.262661281475522079.JavaMail.jira@thor>
Date: Tue, 10 Aug 2010 17:25:22 -0400 (EDT)
From: "Luke Lu (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Subject: [jira] Commented: (MAPREDUCE-1881) Improve
 TaskTrackerInstrumentation
In-Reply-To: <20831650.97611277001383337.JavaMail.jira@thor>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-1881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12897047#action_12897047 ] 

Luke Lu commented on MAPREDUCE-1881:
------------------------------------

Having to route through a specific implementation of composite object could lead to situations that user cannot override without changing library code. Currently, we can measure the overhead of instrumentation by comparing with a noop instrumentation. Forcing it through the composite object incurs overhead of a loop construct and doubles the amount of method calls, which may or may not be acceptable given a user application (it's not you or I who should decide whether it's acceptable or not.)

IMO, you don't even need the composite class in official hadoop source to support multiple listeners, which adds minor convenience as well as maintenance burden to Hadoop developers. The user instrumentation feature is supposedly only for experts who knows how to write a more complex instrumentation class than a trivial composite class.

> Improve TaskTrackerInstrumentation
> ----------------------------------
>
>                 Key: MAPREDUCE-1881
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1881
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Matei Zaharia
>            Assignee: Matei Zaharia
>            Priority: Minor
>         Attachments: mapreduce-1881-v2.patch, mapreduce-1881-v2b.patch, mapreduce-1881.patch
>
>
> The TaskTrackerInstrumentation class provides a useful way to capture key events at the TaskTracker for use in various reporting tools, but it is currently rather limited, because only one TaskTrackerInstrumentation can be added to a given TaskTracker and this objects receives minimal information about tasks (only their IDs). I propose enhancing the functionality through two changes:
> # Support a comma-separated list of TaskTrackerInstrumentation classes rather than just a single one in the JobConf, and report events to all of them.
> # Make the reportTaskLaunch and reportTaskEnd methods in TaskTrackerInstrumentation receive a reference to a whole Task object rather than just its TaskAttemptID. It might also be useful to make the latter receive the task's final state, i.e. failed, killed, or successful.
> I'm just posting this here to get a sense of whether this is a good idea. If people think it's okay, I will make a patch against trunk that implements these changes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.