hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Lu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6685) Change the generic serialization framework API to use serialization-specific bytes instead of Map<String,String> for configuration
Date Thu, 18 Nov 2010 23:05:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933599#action_12933599
] 

Luke Lu commented on HADOOP-6685:
---------------------------------

Regarding patch v6:

h4. About thrift:

Since the maven artifact for thrift is not yet ready and that no existing code (other than
new test code) depends on thrift. Maybe we can split the thrift portion out to a second patch,
which can also serve as an example of how to add a new serialization scheme. Otherwise, we'd
have to make changes to all the projects that depend on hadoop-common now and later. Basically
1 additional patch versus 2n patches (n is number of projects depending on hadoop-common.)

h4. Code nits:

* I see code throwing RuntimeException("Error in deserialization") etc (e.g. in DeserializationRawComparator.java).
It'd be nicer to have a more specific runtime exception i.e. SerializationException for easier
matching in various situations (e.g. {{@Test(expected=SerializationException.class)}} etc.)
* in WritableSerialization.java#getRawComparator, you can use Class<WritableComparable<Writable>>
to avoid the rawtype warning.
* I didn't see any tests for Text#readRawString and writeRawString.



> Change the generic serialization framework API to use serialization-specific bytes instead
of Map<String,String> for configuration
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-6685
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6685
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>             Fix For: 0.22.0
>
>         Attachments: libthrift.jar, serial.patch, serial4.patch, serial6.patch, SerializationAtSummit.pdf
>
>
> Currently, the generic serialization framework uses Map<String,String> for the
serialization specific configuration. Since this data is really internal to the specific serialization,
I think we should change it to be an opaque binary blob. This will simplify the interface
for defining specific serializations for different contexts (MAPREDUCE-1462). It will also
move us toward having serialized objects for Mappers, Reducers, etc (MAPREDUCE-1183).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message