hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Lucene-hadoop Wiki] Update of "HadoopMapReduceSecondaryData" by JackHebert
Date Wed, 02 May 2007 21:45:16 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.

The following page has been changed by JackHebert:
http://wiki.apache.org/lucene-hadoop/HadoopMapReduceSecondaryData

New page:
== Programmatically Reading from DFS ==

Occasionally it become desirable to read an extended amount on data from DFS to configure
a mapper/reducer. This can be done by reading a sequenceFile from the DFS during the configure()
call to the mapper/reducer.

{{{
FileSystem fs = FileSystem.get(conf);
Path path = new Path("/user/jhebert/out/part-00000");
SequenceFile.Reader reader = new SequenceFile.Reader(fs, path, conf);

Text key = new Text();
Text value = new Text();
while (true) {
    reader.next(key, value);
    if (key.toString().equals(""))
        break;
    // Do something useful with the data.
}		
}}}


Mime
View raw message