hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Loddengaard (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3788) Add serialization for Protocol Buffers
Date Thu, 11 Sep 2008 02:42:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12630072#action_12630072

Alex Loddengaard commented on HADOOP-3788:

Tom, thanks for correcting my JIRA misuse.

PBs do not provide a mechanism to limit the amount of data read from a stream, so your solution
of breaking key, value pairs into two streams is the approach we should take.  I'll make this

As for your other comments, I included the *Tracker classes and TestPBHadoopStreams to prove
that streams have extra binary data.  I was planning on removing these classes once it was
widely understood that the streams had extra data, but I'll keep them around and modify them

Good idea on keeping a single Builder instance per deserializer.  That's a much better solution.

I'll get working on this and submit a patch.  Thanks again, Tom!

> Add serialization for Protocol Buffers
> --------------------------------------
>                 Key: HADOOP-3788
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3788
>             Project: Hadoop Core
>          Issue Type: Wish
>          Components: examples, mapred
>    Affects Versions: 0.19.0
>            Reporter: Tom White
>            Assignee: Alex Loddengaard
>             Fix For: 0.19.0
>         Attachments: hadoop-3788-v1.patch, protobuf-java-2.0.1.jar
> Protocol Buffers (http://code.google.com/p/protobuf/) are a way of encoding data in a
compact binary format. This issue is to write a ProtocolBuffersSerialization to support using
Protocol Buffers types in MapReduce programs, including an example program. This should probably
go into contrib. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message