Return-Path: Delivered-To: apmail-hadoop-core-dev-archive@www.apache.org Received: (qmail 36658 invoked from network); 9 Dec 2008 20:54:10 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 9 Dec 2008 20:54:10 -0000 Received: (qmail 73704 invoked by uid 500); 9 Dec 2008 20:54:19 -0000 Delivered-To: apmail-hadoop-core-dev-archive@hadoop.apache.org Received: (qmail 73669 invoked by uid 500); 9 Dec 2008 20:54:19 -0000 Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-dev@hadoop.apache.org Delivered-To: mailing list core-dev@hadoop.apache.org Received: (qmail 73652 invoked by uid 99); 9 Dec 2008 20:54:19 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Dec 2008 12:54:19 -0800 X-ASF-Spam-Status: No, hits=-4.0 required=10.0 tests=RCVD_IN_DNSWL_MED X-Spam-Check-By: apache.org Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Dec 2008 20:54:05 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 8ADEA234C2BC for ; Tue, 9 Dec 2008 12:53:44 -0800 (PST) Message-ID: <748272456.1228856024567.JavaMail.jira@brutus> Date: Tue, 9 Dec 2008 12:53:44 -0800 (PST) From: "stack (JIRA)" To: core-dev@hadoop.apache.org Subject: [jira] Commented: (HADOOP-4797) RPC Server can leave a lot of direct buffers In-Reply-To: <1853322372.1228530224244.JavaMail.jira@brutus> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654956#action_12654956 ] stack commented on HADOOP-4797: ------------------------------- Raghu: There is one on line #60 ('lager') but is so minor it does not merit regen of patches. Ran a little loading test w/ patch installed in a 0.19 pedigree hadoop and nothing obviously broken. +1 again. > RPC Server can leave a lot of direct buffers > --------------------------------------------- > > Key: HADOOP-4797 > URL: https://issues.apache.org/jira/browse/HADOOP-4797 > Project: Hadoop Core > Issue Type: Bug > Components: ipc > Affects Versions: 0.17.0 > Reporter: Raghu Angadi > Assignee: Raghu Angadi > Fix For: 0.18.3, 0.19.1, 0.20.0 > > Attachments: HADOOP-4797-branch-18.patch, HADOOP-4797-branch-18.patch, HADOOP-4797-branch-18.patch, HADOOP-4797.patch > > > RPC server unwittingly can soft-leak direct buffers. One observed case is that one of the namenodes at Yahoo took 40GB of virtual memory though it was configured for 24GB memory. Most of the memory outside Java heap expected to be direct buffers. This shown to be because of how RPC server reads and writes serialized data. The cause and proposed fix are in following comment. > -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.