hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-1841) IPC server should write repsonses asynchronously
Date Wed, 10 Oct 2007 05:16:52 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

dhruba borthakur updated HADOOP-1841:

    Attachment: asyncRPC.patch

Hi Enis,  I took the liberty of writing some code to do asynchronous RPC responses from the
server. This patch has the following:

1. There is *one* additional Responder thread, whose job is to write the RPC responses back
to the RPC clients. It does non-blocking IO on the socket. Because it does non-blocking IO,
only one Responder thread is good enough.

2. The RPC handler threads queues the responses of the RPCs into buffers associated with the
response-channel. These responses are flushed to the client by the Responder thread described

3. A test case that creates 50 threads and issues concurrent RPCs. Each of these RPCs transfers
400K of data from client to server and back from server to client.

This is very preliminary code but I would like to enhance the test to measure whether this
change is making any difference to performance.

> IPC server should write repsonses asynchronously
> ------------------------------------------------
>                 Key: HADOOP-1841
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1841
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: ipc
>            Reporter: Doug Cutting
>            Assignee: Enis Soztutar
>         Attachments: asyncRPC.patch
> Hadoop's IPC Server currently writes responses from request handler threads using blocking
writes.  Performance and scalability might be improved if responses were written asynchronously.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message