hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7557) Make IPC header be extensible
Date Wed, 12 Oct 2011 22:25:12 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13126209#comment-13126209
] 

Doug Cutting commented on HADOOP-7557:
--------------------------------------

> The extra field (protocolKind) I added was to distinguish between the multiple PRC Engines
and catch bugs when there is a mismatch - this should have been done when we added the multiple
RPC engines.

I disagree.  We need to have two engines: present and future.  A developer should be able
to configure things to use future, but only present should ever be used in production.  Our
wire format should not be engineered around supporting multiple wire formats in production.
 Rather a given release should support a single wire format.  We'll someday switch to future,
which supports inter-version and inter-language compatibility, security, has good performance,
etc.  At that point we discard present.

> Essentially you are arguing that IpcHeader does not need to be extensible and that essentially
we should merge the "hrpc" header and the ipcHeader.

No, the wire format of headers is a property of the protocol engine, not a universal thing.
 The existing AvroRpcEngine is a work in progress.  It piggybacks on WritableRpc for its wire
protocol as its initial goal was just to debug just message serialization, then subsequently
change the full protocol wire format, including headers, security, etc.  Work on AvroRpcEngine
stopped when e14 vetoed this proposed present/future incremental approach.  If that work were
to continue, I might next try to switch AvroRpcEngine to a different underlying network protocol
(e.g., HTTP, Avro's SASL profile, or something new) that's language independent, has other
implementations, etc.  Or work could be done on switching from Avro reflect to specific. 
But we shouldn't extend the legacy RPC format to also handle the future format, otherwise
we're stuck with the legacy forever.
                
> Make  IPC  header be extensible
> -------------------------------
>
>                 Key: HADOOP-7557
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7557
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Sanjay Radia
>            Assignee: Sanjay Radia
>         Attachments: HADOOP-7557.patch, IpcHeader.proto, ipcHeader1.patch, ipcHeader2.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message