hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4077) Issues while using Hadoop Streaming job
Date Wed, 28 Mar 2012 15:55:28 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240491#comment-13240491
] 

Robert Joseph Evans commented on MAPREDUCE-4077:
------------------------------------------------

It is not a -D option.  

-D conf=file.conf

will not work. It has to be -conf file.conf. But all generic options must appear on the command
line before any of the non-generic options like -input, -output, etc.

The Generic Options Supported message is correct, but it does not indicate where those options
are allowed on the command line and where they are not allowed.  If you want to reopen the
ticket to improve the message that would be great.


 
                
> Issues while using Hadoop Streaming job
> ---------------------------------------
>
>                 Key: MAPREDUCE-4077
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4077
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.1
>            Reporter: Devaraj K
>            Assignee: Devaraj K
>
> When we use -file option it says deprecated and use -files.
> {code:xml}
> linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar 
> ../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop 
> -output /test/output/3 -mapper cat -reducer wc -file hadoop
> 02/02/19 10:55:51 WARN streaming.StreamJob: -file option is deprecated, please use generic
option -files instead.
> {code}
> But when we use -files option, it says unrecognized option.
> {code:xml}
> linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar 
> ../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop -output

> /test/output/3 -mapper cat -reducer wc -files hadoop
> 02/02/19 10:56:42 ERROR streaming.StreamJob: Unrecognized option: -files
> Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
> {code}
> When we use -archives option,  it says unrecognized option.
> {code:xml}
> linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar 
> ../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop -output

> /test/output/3 -mapper cat -reducer wc -archives testarchive.rar
> 02/02/19 11:05:43 ERROR streaming.StreamJob: Unrecognized option: -archives
> Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
> {code}
> But in the options it will display the usage of the -archives.
> {code:xml}
> linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop jar 
> ../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop -output

> /test/output/3 -mapper cat -reducer wc -archives testarchive.rar
> 02/02/19 11:05:43 ERROR streaming.StreamJob: Unrecognized option: -archives
> Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
> ..........
> ..........
> -libjars <comma separated list of jars>    specify comma separated jar files to
include in the classpath.
> -archives <comma separated list of archives>    specify comma separated archives
to be unarchived on the compute machines.
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message