hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1841) IPC server should write repsonses asynchronously
Date Wed, 10 Oct 2007 17:18:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533804
] 

dhruba borthakur commented on HADOOP-1841:
------------------------------------------

Please let me explain why i think this is a necessary pre-requisite for increasing namenode
transaction performance.

The namenode has 40 threads serving client requests. A majority of these threads are waiting
for a transaction to complete, i.e. a disk sync. The disk-sync times are pretty high on loaded
systems. If we can avoid doing the edit-log sync from the IPC handler thread, instead do it
from a separate  Sycner thread, we can free up the IPC handler thread to process new client
requests. To do this, we need the ability to be able to respond to a RPC from a thread that
is separate from the IPC handler thread.

Enis: Thanks for looking over the patch I submitted and making enhancements to it.

> IPC server should write repsonses asynchronously
> ------------------------------------------------
>
>                 Key: HADOOP-1841
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1841
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: ipc
>            Reporter: Doug Cutting
>            Assignee: dhruba borthakur
>         Attachments: asyncRPC.patch, asyncRPC.patch
>
>
> Hadoop's IPC Server currently writes responses from request handler threads using blocking
writes.  Performance and scalability might be improved if responses were written asynchronously.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message