hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9163) The rpc msg in ProtobufRpcEngine.proto should be moved out to avoid an extra copy
Date Wed, 26 Dec 2012 23:44:15 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13539751#comment-13539751
] 

Sanjay Radia commented on HADOOP-9163:
--------------------------------------

Todd, I was planning to change Hadoop rpc to use  CodedOutputStream to reduce copies; this
requires no changes in protocol and is merely code change in how messages are serialized.
However, the nested request field will cause an extra copy.
See HBases's ProtobufRpcCLientEngine#constructRpcRequest
{code}
RpcRequestBody constructRpcRequest(Method method, Object[] params) throws ServiceException
{
     RpcRequestBody.Builder builder = RpcRequestBody.newBuilder();
...
        param = (Message)params[0];
...
      builder.setRequest(param.toByteString());
...
    }
{code}
The param.toByteString will cause the rpc request message to be serialized as bytes. This
will be serialized once again when RpcRequestBody is serialized later. (There is similar code
in Hadoop RPC since HBase rpc was copied from Hadoop rpc as far as I understand.)
Am i mistaken here?

I have tried to see if could have fixed this in a compatible way by setting the type of Request
to be "message" instead of "bytes" as this would not change the protocol since bytes and message
are the same; however I cannot define a field to be "message" in a .proto file.
Hence my suggesting to move it out. I have not marked this as jira as incompatible yet since
I am still exploring a couple of options.
                
> The rpc msg in  ProtobufRpcEngine.proto should be moved out to avoid an extra copy
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-9163
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9163
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Sanjay Radia
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message