hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-433) Better access to the RecordReader
Date Fri, 11 Aug 2006 19:39:14 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-433?page=comments#action_12427599 ] 
            
Doug Cutting commented on HADOOP-433:
-------------------------------------

Are you proposing that we should add a getSplit() method to RecordReader?  Then the progress
wrapper could delegate to provide the original split.  Would that suffice?

BTW, do we have an issue anywhere to add a generic Split interface?  I couldn't find one.

> Better access to the RecordReader
> ---------------------------------
>
>                 Key: HADOOP-433
>                 URL: http://issues.apache.org/jira/browse/HADOOP-433
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.5.0
>            Reporter: Benjamin Reed
>            Priority: Minor
>
> The record reader has access to the FileSplit which can in turn have information that
is useful to the Mapper. For example, Map processing may vary according to file name or attributes
associated with a file. Unfortunately, even using a MapRunner you only have access to the
progress wrapper of the RecordReader. To get access to the real record reader I had to use
a thread local variable which I set in RecordReader.getNext(). It would be much nicer if you
could get a reference to the real RecordReader from the RecordReader passed to MapRunner.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message