hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: set replication of output to 1 in CDH3
Date Wed, 22 Aug 2012 16:41:06 GMT
Hi,

This is your issue:

> 12/08/22 18:04:42 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.

For using the -D parameters automatically, your application needs to
implement the Tool interface. See
http://archive.cloudera.com/cdh/3/hadoop/api/org/apache/hadoop/util/Tool.html
for the API and an example.

On Wed, Aug 22, 2012 at 9:39 PM, Matthias Kricke
<matthias.mk.kricke@gmail.com> wrote:
> Dear mailing list,
>
> I want to set the replication of my output file to 1 for clouderas hadoop 3.
> Some web research told me to do the following:
> hadoop jar fu.jar com.bar -D dfs.replication=1 input output
>
> I adapted it a bit, because my jar has a specified main class. I do
> something like that:
> hadoop jar fu.jar -D dfs.replication=1 input output
>
> When I now execute this, I get the following error:
>
> 12/08/22 18:04:42 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the same.
> 12/08/22 18:04:43 INFO mapred.JobClient: Cleaning up the staging area
> someServerAndFileAdress
> Exception in thread "main"
> org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does
> not exist: someServerAndFileAdress/-D
> at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:231)
> at
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:248)
> at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:944)
> at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:961)
> at org.apache.hadoop.mapred.JobClient.access$500(JobClient.java:170)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:880)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1127)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:476)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:506)
> at
> com.unister.semweb.preprocessingMapperStyle.core.StartMapper.main(StartMapper.java:43)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
>
> I would appreciate your help.
>
> Greetings,
> MK
>



-- 
Harsh J

Mime
View raw message