hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <o...@yahoo-inc.com>
Subject Re: running new jobs
Date Mon, 08 Jan 2007 06:17:17 GMT

On Jan 5, 2007, at 3:58 AM, Torsten Curdt wrote:

> A few things that aren't really clear to me yet ...hadoop is  
> deployed and I want to schedule a new job. Let's say it is written  
> in java. Will hadoop distribute the classes so the job is available  
> on all nodes? Or do I have to make it is deployed everywhere.

If your classes are put together in a jar archive and the JobConf is  
told to use that jar, then jar is distributed for you. This is what  
the examples do.

> Also: there are python examples available. Will the python script  
> be distributed when scheduled and an external process will be  
> called to execute the job? IIUC this works by piping the data  
> through the process ...so in theory it could be any language - is  
> that correct? Or is jython involved somewhere?

The python example I wrote worked by using jython and building a jar  
file. From that point, it worked just like Java.

-- Owen

View raw message