Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@apache.org
Date: Tue, 15 Sep 2015 15:45:46 +0000 (UTC)
From: "Josh Elser (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.12668053.1378908497000.343190.1442331946765@Atlassian.JIRA>
In-Reply-To: <JIRA.12668053.1378908497000@Atlassian.JIRA>
References: <JIRA.12668053.1378908497000@Atlassian.JIRA>
 <JIRA.12668053.1378908497886@arcas>
Subject: [jira] [Commented] (ACCUMULO-1708) Error during minor compaction
 left tserver in bad state
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/ACCUMULO-1708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14745624#comment-14745624 ] 

Josh Elser commented on ACCUMULO-1708:
--------------------------------------

bq. I'm kidding. When I mean you, I actually mean Josh. 

I'm a fan of all things behooved.

bq. I think the best course of action is to modify Accumulo code to catch Error and halt for the threads it creates (maybe use thread groups to do this)

If you s/Error/OutOfMemoryError/, I think I'd agree with your approach. I've never seen a JVM that throws an OOME (for heap or native thread reasons) that actually recovered successfully. While I'm not a big fan of a process hanging itself, I think this case it might be a good idea.

> Error during minor compaction left tserver in bad state
> -------------------------------------------------------
>
>                 Key: ACCUMULO-1708
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1708
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.4.0
>            Reporter: Keith Turner
>            Priority: Critical
>             Fix For: 1.8.0
>
>         Attachments: ThreadTest.java
>
>
> A tserver experienced a OOME during minor compaction.  This OOME was thrown because java could not create a native thread.  Minor compactions only catch declared exceptions and RuntimeExceptions.  This left the system in a state where the compaction was not running but the tserver thought it was.  This cause"flush -w" to hang and prevented the tserver from reclaiming memory.
> For whatever reason the OOME handler that kills the process did not kick in (seems it only kicks in w/ OOME related to heap allocation).


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)