hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1841) IPC server should write repsonses asynchronously
Date Wed, 10 Oct 2007 17:18:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533804

dhruba borthakur commented on HADOOP-1841:

Please let me explain why i think this is a necessary pre-requisite for increasing namenode
transaction performance.

The namenode has 40 threads serving client requests. A majority of these threads are waiting
for a transaction to complete, i.e. a disk sync. The disk-sync times are pretty high on loaded
systems. If we can avoid doing the edit-log sync from the IPC handler thread, instead do it
from a separate  Sycner thread, we can free up the IPC handler thread to process new client
requests. To do this, we need the ability to be able to respond to a RPC from a thread that
is separate from the IPC handler thread.

Enis: Thanks for looking over the patch I submitted and making enhancements to it.

> IPC server should write repsonses asynchronously
> ------------------------------------------------
>                 Key: HADOOP-1841
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1841
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: ipc
>            Reporter: Doug Cutting
>            Assignee: dhruba borthakur
>         Attachments: asyncRPC.patch, asyncRPC.patch
> Hadoop's IPC Server currently writes responses from request handler threads using blocking
writes.  Performance and scalability might be improved if responses were written asynchronously.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message