hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2660) Turn off TestDFSOverAvroRpc
Date Tue, 13 Dec 2011 18:48:30 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13168603#comment-13168603
] 

Doug Cutting commented on HDFS-2660:
------------------------------------

> In order to support Avro, the right way to do it is, change the getServer() calls and
addProtocol() calls to use WireProtocol and no longer just setting the RPC engine.

What do you mean by WireProtocol here?  The protobuf-generated Java interface?  That contains,
e.g., fields that are not meant to be transmitted but are not declared transient.  In other
words, it's not a very good specification of a wire protocol.

Avro actually includes code that knows how to serialize protobuf structs but doesn't yet include
protobuf-specific code for protocols (http://s.apache.org/MZT).  That could be added so that
protobuf becomes the 'serialization-independent' Hadoop protocol specification instead of
using Java interfaces as we have before.  I'd welcome that contribution to Avro.  Then RPCEngine
might then be modified to accept protobuf ServiceDescriptors to specify protocols rather than
Java interfaces.  Or we could abandon serialization-independent RPC as an architectural goal
and remove the code related to it.
                
> Turn off TestDFSOverAvroRpc
> ---------------------------
>
>                 Key: HDFS-2660
>                 URL: https://issues.apache.org/jira/browse/HDFS-2660
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: test
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>             Fix For: 0.24.0
>
>         Attachments: HDFS-2660.patch, HDFS-2660.txt
>
>
> With HDFS-2647, protobuf based RPCs are enabled for some of the protocol. With this,
Avro RPC based protocol is not working. Avro based RPC needs to be turned on similar to how
protobuf based RPCs are done. Until such a time, I propose turning off the test.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message