hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Update of "RandomWriter" by OwenOMalley
Date Wed, 28 Jun 2006 21:43:35 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by OwenOMalley:
http://wiki.apache.org/lucene-hadoop/RandomWriter

------------------------------------------------------------------------------
  This example uses a useful pattern for dealing with Hadoop's constraints on !InputSplits.
Since each input split can only consist of a file and byte range and we want to control how
many maps there are (and we don't really have any inputs), we create a directory with a set
of artificial files, each of which contains the filename that we want a given map to write
to. Then, using the text line reader and this "fake" input directory, we generate exactly
the right number of maps. Each map gets a single record that is the filename, to which it
is supposed to write its output. 
  
  To run the example, the command syntax is[[BR]]
- bin/hadoop org.apache.hadoop.examples.RandomWriter <out-dir> [<configuration file>]
+ bin/hadoop jar hadoop-*-examples.jar randomwriter <out-dir> [<configuration file>]
  

Mime
View raw message