hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2975) IPC server should not allocate a buffer for each request
Date Tue, 03 Jun 2008 17:38:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12602002#action_12602002
] 

Doug Cutting commented on HADOOP-2975:
--------------------------------------

1MB per connection with a limit of 4000 connections (by default) means that we'll use 4GB
of heap for RPC buffers.  That's clearly too much.  Except block reports, most RPC requests
are probably in the 1kB range.

> IPC server should not allocate a buffer for each request
> --------------------------------------------------------
>
>                 Key: HADOOP-2975
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2975
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: ipc
>    Affects Versions: 0.16.0
>            Reporter: Hairong Kuang
>         Attachments: Hadoop-2975-v1.patch, Hadoop-2975-v2.patch
>
>
> Currently the IPC server allocates a buffer for each incoming request. The buffer is
thrown away after the request is serialized. This leads to very inefficient heap utilization.
It would be nicer if all requests from one connection could share a same common buffer since
the ipc server has only one request is being read from a socket at a time.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message