hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J Chouraria (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1360) Record skipping should work with more serializations
Date Sat, 26 Feb 2011 06:47:21 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12999728#comment-12999728

Harsh J Chouraria commented on MAPREDUCE-1360:

This sounded interesting for me to work upon. So here's some questions after a brief investigation:

I investigated what goes on inside of the SkippingRecordReader but couldn't find any area
that gets Writable limited. The sequence files for skipped records are created with the Key/Value
classes which are then used to load their acceptable serialization classes. Which part of
the skipping framework is Writable limited now?

> Record skipping should work with more serializations
> ----------------------------------------------------
>                 Key: MAPREDUCE-1360
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1360
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Aaron Kimball
> Record skipping currently supports WritableSerialization, but cannot handle non-class-based
serialization systems (e.g., AvroSerialization). The record skipping mechanism should be made
compatible with the metadata-based serialization configuration.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message