hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clément MATHIEU <clem...@unportant.info>
Subject How to download TaskLog.PROFILE files ?
Date Sun, 22 Mar 2015 14:44:45 GMT
Hi,

I use HPROF to profile my mapreduce / MR2 jobs and I am looking for a
way to automatically download the "profile.out" files created by the
profiled tasks. I intend to do it from the driver once the job is
finished so a Java API is welcome but I am also fine using one of the
REST APIs.

LogCLIHelpers shown me how it can be done, but it seems a bit convoluted
and relies on private API. REST APIs have logsLink entries. However the
link points to web page making it difficult to get the file.

Any idea or code sample ?

I would also be interested by a command line tool allowing to do the
same thing (without having to parse "yarn logs") 

Thanks.

Clément MATHIEU



Mime
View raw message