hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Binglin Chang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9151) Include RPC error info in RpcResponseHeader instead of sending it separately
Date Sat, 05 Jan 2013 07:56:17 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13544625#comment-13544625
] 

Binglin Chang commented on HADOOP-9151:
---------------------------------------

IMO, current RPC protocol has lot of issues, some can be solved without breaking compatibility,
some can not, this JIRA(maybe not worth to change) is just one of them, and many others in
HADOOP-8990 are not complete either. How about first propose a complete change solution, with
performance test results, then evaluate whether it is worth or not?
In addition, list some of the unresolved or potential issues of RPC protocol:
# We can't assume msg.writeDelimitedTo avoids buffer copy, I read some protobuf code, in fact
it create a huge buffer to do serialization(basically the same as "message.toByteArray()"),
unfortunately "writeDelimitedTo" is used everywhere, and compare to buffer allocation, zeroing,
some extra buffer copy is hardly performance bottleneck. So some statement in HADOOP-9163
and HADOOP-8084 is not valid. If we truly want a zero copy implementation, perhaps we should
reuse CodedOutputStream throughout RPC code rather than each time create a new one, this probably
give more performance gains than avoid buffer copy. Whether or not, first some test results
is required before change.
# Length prefix have no convention at all, we have 4 byte, varint prefixes, and some has -1
to denote special meaning(null string or ping id), some not, all need additional code to deal
with.
# Protobuf style varint is not convenient for writing aio, non-blocking client/server(still
possible), so is -1 prefix, if aio rpc is on the road map, consider unified to 4 byte int
prefix with no special meaning.
# Merge request/response into single proto has one extra benefits, the whole protocol is now
truly compatible, new "sub packets" can be freely added or removed, whether it is a new header
or error message, or some new packet type to implement RPC_CONTINUATION_PACKET?

                
> Include RPC error info in RpcResponseHeader instead of sending it separately
> ----------------------------------------------------------------------------
>
>                 Key: HADOOP-9151
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9151
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Sanjay Radia
>            Assignee: Sanjay Radia
>         Attachments: HADOOP-9151.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message