hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-967) TaskTracker does not need to fully unjar job jars
Date Fri, 13 Nov 2009 23:09:39 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Todd Lipcon updated MAPREDUCE-967:

    Attachment: mapreduce-967.txt

New patch for the new version of HADOOP-6346. This one does *not* move the RunJar class to
mapreduce, since we determined over in that issue that it isn't the best course of action.

One question for reviewer: the constant for the new configuration key is in JobContext, whereas
the default is in JobConf. I was following some other examples from the code, but it seems
a little bit messy here. Where are the right places to add new configuration parameters that
work in both APIs?

> TaskTracker does not need to fully unjar job jars
> -------------------------------------------------
>                 Key: MAPREDUCE-967
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-967
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: mapreduce-967-branch-0.20.txt, mapreduce-967.txt, mapreduce-967.txt,
> In practice we have seen some users submitting job jars that consist of 10,000+ classes.
Unpacking these jars into mapred.local.dir and then cleaning up after them has a significant
cost (both in wall clock and in unnecessary heavy disk utilization). This cost can be easily

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message