hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jie Li (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-4365) Shipping Profiler Libraries by DistributedCache
Date Mon, 25 Jun 2012 07:20:42 GMT
Jie Li created MAPREDUCE-4365:

             Summary: Shipping Profiler Libraries by DistributedCache
                 Key: MAPREDUCE-4365
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4365
             Project: Hadoop Map/Reduce
          Issue Type: New Feature
    Affects Versions: 1.0.3
            Reporter: Jie Li

Hadoop profiling is great for performance tuning and debugging, but currently we can only
use Java built-in profilers such as HProf, and for other profilers we need to install them
on all slave nodes first, which is inconvenient for large clusters and sometimes impossible
for production clusters. 

Supporting shipping profiler libraries using DistributedCache will solve this problem. For
example, in mapred.task.profile.params, we specify a profiler library from the DistributedCache
using special place holders such as <foo.jar>, and Hadoop can look at the DistributedCache
to replace <foo.jar> with the localized path before launching the child jvm.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message