hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pete Wyckoff (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HADOOP-4192) Class <? extends T> Deserializer.getRealClass() method to return the actual class of the objects from a deserializer
Date Thu, 18 Sep 2008 03:13:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Pete Wyckoff resolved HADOOP-4192.
----------------------------------

    Resolution: Invalid

should be attacking this the other way around by having a way of getting serialization context
info - the serialization class and the class to be deserialized and pass these down.


> Class <? extends T> Deserializer.getRealClass() method to return the actual class
of the objects from a deserializer
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-4192
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4192
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Pete Wyckoff
>
> Note: this use case is completely for non-self describing files with Serialization framework
records.  If the Serialization Class and the actual type of records to be deserialized is
configured higher up through the JobConf.  
> It is motivated by the need to create a generic FlatFileDeserializerRecordReader that
can be configued to use any Serialization implementation through the JobConf.
> Since A deserializer can return a subtype of the type it is instantiated to return, we
can create generic Deserializers for a base type - e.g., Writable, Record, Thrift.Tbase where
the RecordReader need not be specific to any of them.  
> In which case,to implement RecordReader.getValueClass();, the generic RecordReader really
needs to query that from the Deserializer.
> And since this RecordReader is generic even ithe Serialization Implementation it is going
to use should come from the JobConf as should the actual specific class being Deserialized.
e.g., Record/MyUserIDRecord, Writable/LongWritable.
> The RecordReader would need to know how the Serialization and Deserializer get their
configuration info to implement getValueClass().
> A much cleaner way is to implement getRealClass I think.
>  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message