hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4802) RPC Server send buffer retains size of largest response ever sent
Date Mon, 08 Dec 2008 19:16:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654526#action_12654526
] 

Raghu Angadi commented on HADOOP-4802:
--------------------------------------

I was thinking on the same lines. Allocation per request would not hurt (mostly because various
other extra costs). Coule of improvements by subclassing BAOS:

   # Keep a fixed buffer 10kb. inside reset() replace the previous buffer with this. This
avoids allocation for most RPCs.
   # extra benefit : this would allow us to avoid a copy if the response could be written
to socket in-line (common case). Right now server always copies (through asByteArray()).

Note that this jira will increase the buffer copies for large (multi-MB) responses because
of copy each time buffer expands. (Partly negated by copies reduced by HADOOP-4797, since
it takes multiple write() invocations to write all the data. in jdk, each write copies all
the data!). A future improvement could be to write our own stream that uses a list of buffers.


> RPC Server send buffer retains size of largest response ever sent 
> ------------------------------------------------------------------
>
>                 Key: HADOOP-4802
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4802
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: ipc
>    Affects Versions: 0.18.2, 0.19.0
>            Reporter: stack
>         Attachments: 4802.patch
>
>
> The stack-based ByteArrayOutputStream in Server.Hander is reset each time through the
run loop.  This will set the BAOS 'size' back to zero but the allocated backing buffer is
unaltered.  If during an Handlers' lifecycle, any particular RPC response was fat -- Megabytes,
even -- the buffer expands during the write to accommodate the particular response but then
never shrinks subsequently.  If a hosting Server has had more than one 'fat payload' occurrence,
the resultant occupied heap can provoke memory woes (See https://issues.apache.org/jira/browse/HBASE-900?focusedCommentId=12654009#action_12654009
for an extreme example; occasional payloads of 20-50MB with 30 handlers robbed the heap of
700MB).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message