hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghu Angadi <rang...@apache.org>
Subject Re: purpose of JobTracker copying job.jar
Date Mon, 09 May 2011 07:55:01 GMT
On Wed, Apr 27, 2011 at 11:20 AM, Arun C Murthy <acm@yahoo-inc.com> wrote:

> So that it can instruct TTs to copy it from the system-dir for actually
> running jobs..
>

Thanks Arun.
Task trackers copy the job jar from HDFS (placed by client).
JobTracker copying the same file to its local filesystem seems to be
unnecessary.. This could have very noticeable effects on the central server.

I commented out JobInProgress.java:266 on 0.20 trunk, and my test jobs run
fine. I will try to double check what I am missing.

Raghu.


> hope that helps,
> Arun
>
>
> On Apr 26, 2011, at 11:57 AM, Raghu Angadi wrote:
>
>
>> Job tracker copies job jar from mapred system directory on HDFS to its
>> local file system ( ${mapred.local.dir}/jobTracker ).
>>
>> What is the purpose of copying this file? It does not run any user code.
>>
>> thanks,
>> Raghu.
>>
>>
>

Mime
View raw message