hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley" <...@yahoo-inc.com>
Subject Re: Submitting jobs into Hadoop
Date Fri, 13 Jul 2007 01:46:35 GMT
On Jul 12, 2007, at 2:10 PM, Doug Cutting wrote:

> The job's jar file is unpacked in the directory that tasks are  
> connected to when they run.  The classpath of the task JVM includes  
> all jars in the lib/ directory in the job's jar, the classes/  
> directory from the jar, and the top-level directory of the jar.

Just a few more details:
   1. The JobClient copies the jar from the local file system into  
HDFS under the "system" directory.
   2. The first task for a given job that is run on a TaskTracker  
reads the jar out of HDFS, writes it locally, and
        expands it.
   3. When the job is done, the jar is deleted from all of the  
TaskTrackers and the system directory.

-- Owen

View raw message