hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: DFS and the RecordReader
Date Thu, 06 Dec 2012 23:07:48 GMT
Ah ok, understood what you seem to be looking for.

Lets follow the simple LineReader implementation in that case.

TextInputFormat uses LineRecordReader: [1] - Line 52
LineRecordReader has the calls you look for and wraps over a
LineReader implementation, to take care of reading lines over block
boundaries: [2] - Line 88
LineReader has all the functional code to make it work for anyone
reading lines off of text files: [3]

[1] - http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.2-alpha/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/TextInputFormat.java?view=markup
[2] - http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.2-alpha/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/LineRecordReader.java?view=markup
[3] - http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.2-alpha/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/LineReader.java?view=markup

On Fri, Dec 7, 2012 at 4:17 AM, Jay Vyas <jayunit100@gmail.com> wrote:
> Hmm... so when a record reader calls fs.open(...) , I guess Im looking for
> an example of how the input stream is created... ?



-- 
Harsh J

Mime
View raw message