hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Update of "HadoopStreaming" by TedDunning
Date Fri, 04 Jan 2008 02:02:24 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by TedDunning:
http://wiki.apache.org/lucene-hadoop/HadoopStreaming

------------------------------------------------------------------------------
  
  Usage: $HADOOP_HOME/bin/hadoop jar build/hadoop-streaming.jar [options]
  Options:
-   -input    <path>     DFS input file(s) for the Map step
+   -input    <path>                   DFS input file(s) for the Map step
-   -output   <path>     DFS output directory for the Reduce step
+   -output   <path>                   DFS output directory for the Reduce step
-   -mapper   <cmd>      The streaming command to run
+   -mapper   <cmd|JavaClassName>      The streaming command to run
-   -combiner <cmd>      Not implemented. But you can pipe the mapper output
+   -combiner <JavaClassName>          Combiner has to be a Java class
-   -reducer  <cmd>      The streaming command to run
+   -reducer  <cmd|JavaClassName>      The streaming command to run
-   -file     <file>     File/dir to be shipped in the Job jar file
+   -file     <file>                   File/dir to be shipped in the Job jar file
-   -cluster  <name>     Default uses hadoop-default.xml and hadoop-site.xml
-   -config   <file>     Optional. One or more paths to xml config files
-   -dfs      <h:p>      Optional. Override DFS configuration
+   -dfs    <h:p>|local                Optional. Override DFS configuration
-   -jt       <h:p>      Optional. Override JobTracker configuration
+   -jt     <h:p>|local                Optional. Override JobTracker configuration
+   -additionalconfspec specfile       Optional.
+   -inputformat TextInputFormat(default)|SequenceFileAsTextInputFormat|JavaClassName Optional.
+   -outputformat TextOutputFormat(default)|JavaClassName  Optional.
+   -partitioner JavaClassName         Optional.
+   -numReduceTasks <num>              Optional.
-   -inputreader <spec>  Optional.
+   -inputreader <spec>                Optional.
-   -jobconf  <n>=<v>    Optional.
+   -jobconf  <n>=<v>                  Optional. Add or override a JobConf property
-   -cmdenv   <n>=<v>    Optional. Pass env.var to streaming commands
+   -cmdenv   <n>=<v>                  Optional. Pass env.var to streaming commands
+   -cacheFile fileNameURI
+   -cacheArchive fileNameURI
    -verbose
  
  In -input: globbing on <path> is supported and can have multiple -input

Mime
View raw message