hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (Naga)" <garlanaganarasi...@huawei.com>
Subject RE: Passing Args to Mappers and Reducers
Date Tue, 06 Oct 2015 14:49:54 GMT
Hi Ista,

In general we need to give after specifying the class name like
./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.0.0-SNAPSHOT.jar
sleep -Dmapreduce.job.queuename=default -m 10 -r 1 -mt 5000

I think in your case it should be "yarn  jar /opt/yarn/my_examples/AvgSix.jar <classname
if req> -Dista="666666" /user/yarn/input/samplefile.csv output"

+ Naga
________________________________
From: Naganarasimha G R (Naga)
Sent: Tuesday, October 06, 2015 20:16
To: user@hadoop.apache.org
Subject: RE: Passing Args to Mappers and Reducers

Hi Ista,

In general we need to give after specifying the class name like
./yarn jar ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.0.0-SNAPSHOT.jar
sleep -Dmapreduce.job.queuename=default -m 10 -r 1 -mt 5000

i think in your case it should be
________________________________
From: Istabrak Abdul-Fatah [ifatah@gmail.com]
Sent: Tuesday, October 06, 2015 19:46
To: user@hadoop.apache.org
Subject: Re: Passing Args to Mappers and Reducers

Hi Shahab,
Thx for the quick reply.
Yes, I am using the tool interface and can run the job successfully.
I need to pass some extra args dynamically as part of the invocation.

I have tried adding a space after the -D option and it did not work either (see below).

~]$ yarn -D ista="666666" jar /opt/yarn/my_examples/AvgSix.jar /user/yarn/input/sample.csv
output
Error: Could not find or load main class ista=666666

It does not see to recognize the -D option.

BR

Ista

On Tue, Oct 6, 2015 at 9:26 AM, Shahab Yunus <shahab.yunus@gmail.com<mailto:shahab.yunus@gmail.com>>
wrote:
Are you properly implementing the Tool interface?
https://hadoopi.wordpress.com/2013/06/05/hadoop-implementing-the-tool-interface-for-mapreduce-driver/

Also, there needs to be space between -D and the param name.

Regards,
Shahab

On Tue, Oct 6, 2015 at 9:22 AM, Istabrak Abdul-Fatah <ifatah@gmail.com<mailto:ifatah@gmail.com>>
wrote:
Greetings to all,
Is it possible to pass args to Mapper and Reducers via the command line args.
I need to pass some args upon the MapRed job invocation so that I can pass these args via
the Context object to the Mapper and Reducer code.
I am currently running Hadoop2.7 and tried to pass some args using the -D VM args but it did
not succeed.
Here are some failed code snippets:

~]$ yarn -Dista="666666" jar /opt/yarn/my_examples/AvgSix.jar /user/yarn/input/samplefile.csv
output

~]$ yarn  jar -Dista="666666" /opt/yarn/my_examples/AvgSix.jar /user/yarn/input/samplefile.csv
output

~]$ yarn  jar /opt/yarn/my_examples/AvgSix.jar /user/yarn/input/samplefile.csv output  -Dista="666666"

Please advise.


Thx and BR

Ista



Mime
View raw message