hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-13438) Optimize IPC server protobuf decoding
Date Thu, 28 Jul 2016 22:08:20 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daryn Sharp updated HADOOP-13438:
---------------------------------
    Attachment: HADOOP-13438.patch

Server side only changes to ensure protocol not broken (corresponding client changes forthcoming).

A new RpcWritable class manages the marshaling of writables and PBs.  The Server passes the
ByteBuffer for the read request, instead of the byte[] from the buffer to eliminate a copy.
 The RpcWritables use the ByteBuffer to effectively iterate through the byte[] w/o copying
it.

> Optimize IPC server protobuf decoding
> -------------------------------------
>
>                 Key: HADOOP-13438
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13438
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>         Attachments: HADOOP-13438.patch
>
>
> The current use of the protobuf API uses an expensive code path.  The builder uses the
parser to instantiate a message, then copies the message into the builder.  The parser is
creating multi-layered internally buffering streams that cause excessive byte[] allocations.
> Using the parser directly with a coded input stream backed by the byte[] from the wire
will take a fast-path straight to the pb message's ctor.  Substantially less garbage is generated.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message