hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Dyer (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3788) Add serialization for Protocol Buffers
Date Mon, 03 Nov 2008 20:04:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644794#action_12644794

Chris Dyer commented on HADOOP-3788:

Apologies for just jumping in on this and commenting without much context, but one of my great
hopes for protocol buffer integration in Hadoop is that it will make it easier to integrate
rich, structured data types with non-Java Hadoop clients (specifically c++, but python is
also quite well supported in Hadoop).  Specifically, it would be nice if there could be some
kind of interface definition that would prescribe how non-Java clients could implement mappers
and reducers that would return protocol buffer serializations.  In most cases, I don't think
the framework would even need to know anything about the data types (unless you were implementing
some kind of custom comparator), but if I could easily mix and match languages I would be
a much, much happier camper.  Authors of this patch: can you estimate how difficult incorporating
c++ support would be?

> Add serialization for Protocol Buffers
> --------------------------------------
>                 Key: HADOOP-3788
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3788
>             Project: Hadoop Core
>          Issue Type: Wish
>          Components: contrib/serialization, examples, mapred
>            Reporter: Tom White
>            Assignee: Alex Loddengaard
>             Fix For: 0.20.0
>         Attachments: hadoop-3788-v1.patch, hadoop-3788-v2.patch, protobuf-java-2.0.1.jar
> Protocol Buffers (http://code.google.com/p/protobuf/) are a way of encoding data in a
compact binary format. This issue is to write a ProtocolBuffersSerialization to support using
Protocol Buffers types in MapReduce programs, including an example program. This should probably
go into contrib. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message