hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7557) Make IPC header be extensible
Date Wed, 12 Oct 2011 21:31:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126173#comment-13126173

Sanjay Radia commented on HADOOP-7557:

I am interested in 2 and 3. I haven't stated otherwise. Indeed support for non-java languages
has been a very important motivation for a long time.

Hadoop RPC started to become pluggbale as part of the Avro thrust. Before I started to work
on this recently,  Hadoop rpc already had mulitple RPC engines (Writable, Avro, PB). The extra
field (protocolKind) I added was to distinguish between the multiple PRC Engines and catch
bugs when there is a mismatch - this should have been done when we added the multiple RPC

We agree on the goal: "..select a single message serialization and client/server protocol
that's programming language-independent ..".
Given that the other layers are moving to PB or have already moved to PB (HDFS's data transfer
protocol and MR2 protocols)  why not let the IPC header be in PB? 
Or are you proposing that all the upper protocol layers use a similar technique to the one
you used for the IpcHeader (Map<String,ByteBuffer>) and that we move away from PB and
Avro for Hadoop protocols?

> Make  IPC  header be extensible
> -------------------------------
>                 Key: HADOOP-7557
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7557
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Sanjay Radia
>            Assignee: Sanjay Radia
>         Attachments: HADOOP-7557.patch, IpcHeader.proto, ipcHeader1.patch, ipcHeader2.patch

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message