hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From KayVajj <vajjalak...@gmail.com>
Subject Re: Question regarding hadoop jar command usage
Date Wed, 13 Mar 2013 18:07:41 GMT
Hi Sandy,

I was going through the RunJar source code and the jar executes locally.
When the jar fires a mapreduce job,

the way I create JobConf is

JobConf conf = new JobConf(MyJob.class);
>

Does this set MyJar as the job jar?

Can you explain what is the difference between running an MR job using the
jar command vs mapred job command (looks like the hadoop job command is
deprecated).

Thanks
Kay

On Wed, Mar 13, 2013 at 10:14 AM, Sandy Ryza <sandy.ryza@cloudera.com>wrote:

> Hi Kay,
>
> The jar is just executed locally.  If the jar fires up a mapreduce job and
> sets itself as the job jar, then mapreduce will handle copying it to the
> nodes that will use it.
>
> -Sandy
>
>
> On Wed, Mar 13, 2013 at 9:01 AM, KayVajj <vajjalak009@gmail.com> wrote:
>
>> I have a question regarding the hadoop jar command. In a cluster of say
>> nodes n1,n2...n100
>>
>> the node n1 has jar Myjar on its local file system.
>>
>> If I run the command
>> hadoop jar local/path/to/Myjar Myclass other-args
>>
>> Is the MR job executed just on n1 or any arbitrary node n1..n100?
>>
>> If it is any arbitrary node, since Myjar exists only on n1. Is the jar
>> copied across all nodes on the cluster to run the MR code.
>>
>>
>>
>>
>>
>

Mime
View raw message