Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Message-ID: <747251585.1257535833065.JavaMail.jira@brutus>
Date: Fri, 6 Nov 2009 19:30:33 +0000 (UTC)
From: "Aaron Kimball (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Subject: [jira] Commented: (MAPREDUCE-1131) Using profilers other than hprof
 can cause JobClient to report job failure
In-Reply-To: <1422187703.1256165219529.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774383#action_12774383 ] 

Aaron Kimball commented on MAPREDUCE-1131:
------------------------------------------

Vinod,

Seems reasonable to me. YourKit generates a file in a separate location which developers go and collect manually. I don't have experience with any other profiling tools, so I don't know how many of them do/don't respect a {{file=}} parameter like hprof does.

Looking at its documentation, jprof seems to generate several files in the working directory, but specifying the filename of any one of them is a pain. (It's got a lot of file prefix/suffix arguments.) So it's unlikely that jprof would work with the {{downloadProfile()}} as-written anyway. I don't know what other profiling tools are commonly used, so I don't know what else to check.

So are we sold on "check for hprof and {{file=%s}}" arguments, and select {{downloadProfile()}} based solely on that?

- Aaron


> Using profilers other than hprof can cause JobClient to report job failure
> --------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1131
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1131
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: client
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: MAPREDUCE-1131.2.patch, MAPREDUCE-1131.3.patch, MAPREDUCE-1131.patch
>
>
> If task profiling is enabled, the JobClient will download the {{profile.out}} file created by the tasks under profile. If this causes an IOException, the job is reported as a failure to the client, even though all the tasks themselves may complete successfully. The expected result files are assumed to be generated by hprof. Using the profiling system with other profilers will cause job failure.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.