Return-Path: X-Original-To: apmail-accumulo-notifications-archive@minotaur.apache.org Delivered-To: apmail-accumulo-notifications-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 041F2100A0 for ; Tue, 15 Oct 2013 20:16:50 +0000 (UTC) Received: (qmail 9690 invoked by uid 500); 15 Oct 2013 20:16:47 -0000 Delivered-To: apmail-accumulo-notifications-archive@accumulo.apache.org Received: (qmail 9546 invoked by uid 500); 15 Oct 2013 20:16:47 -0000 Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: jira@apache.org Delivered-To: mailing list notifications@accumulo.apache.org Received: (qmail 9514 invoked by uid 99); 15 Oct 2013 20:16:46 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Oct 2013 20:16:46 +0000 Date: Tue, 15 Oct 2013 20:16:46 +0000 (UTC) From: "Eric Newton (JIRA)" To: notifications@accumulo.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (ACCUMULO-1770) out of memory error on very long running tablet server MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/ACCUMULO-1770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Newton updated ACCUMULO-1770: ---------------------------------- Attachment: FragmentTest.java > out of memory error on very long running tablet server > ------------------------------------------------------ > > Key: ACCUMULO-1770 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1770 > Project: Accumulo > Issue Type: Bug > Components: tserver > Reporter: Eric Newton > Assignee: Eric Newton > Attachments: FragmentTest.java, FragmentTest.java, javamap.png, memory-usage.png, nativemap.png, no-mmap.png, three-day-tserver.png > > > On a large cluster it was noticed that a few of the tablet servers had been pushed into swap. This didn't effect the performance of the server until it ran out of memory, and the process was killed. The gc reports in the debug log showed the system had plenty of heap space for the JVM. The number of threads in the server were not excessive (dozens). This cluster ingests some large values (megabytes). The tablet server had been up for a month prior to running out of memory. MALLOC_ARENA_MAX had already been set to 1. > * Investigate the effect of fragmentation on memory usage for large value inserts. -- This message was sent by Atlassian JIRA (v6.1#6144)