hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pete Wyckoff (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-4166) Deserialize interface to support mechanism for supporting progress method in RecordReader
Date Fri, 12 Sep 2008 17:55:44 GMT
Deserialize interface to support mechanism for supporting progress method in RecordReader
-----------------------------------------------------------------------------------------

                 Key: HADOOP-4166
                 URL: https://issues.apache.org/jira/browse/HADOOP-4166
             Project: Hadoop Core
          Issue Type: Bug
          Components: mapred
            Reporter: Pete Wyckoff


If the deserializer buffers the inputstream, there's no way for the record reader to support
progress or even know when EOF is coming without directly catching EOFException from the deserialize()
method.

so, theRecordReader.next(key,value) method could catch EOFException to return false when needed,but
 the progress method cannot know where in the inputstream it is without an API from the deserializer
in the case where the deserializer does buffering.

An example of this would be implementing LineRecordReader.LineReader as a deserializer since
it reads data into its own buffer.



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message