Return-Path: X-Original-To: apmail-accumulo-notifications-archive@minotaur.apache.org Delivered-To: apmail-accumulo-notifications-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EF938E956 for ; Thu, 28 Feb 2013 21:03:12 +0000 (UTC) Received: (qmail 19188 invoked by uid 500); 28 Feb 2013 21:03:12 -0000 Delivered-To: apmail-accumulo-notifications-archive@accumulo.apache.org Received: (qmail 19131 invoked by uid 500); 28 Feb 2013 21:03:12 -0000 Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: jira@apache.org Delivered-To: mailing list notifications@accumulo.apache.org Received: (qmail 19122 invoked by uid 99); 28 Feb 2013 21:03:12 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Feb 2013 21:03:12 +0000 Date: Thu, 28 Feb 2013 21:03:12 +0000 (UTC) From: "Eric Newton (JIRA)" To: notifications@accumulo.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Resolved] (ACCUMULO-1110) deadlock found in major compaction MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/ACCUMULO-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Newton resolved ACCUMULO-1110. ----------------------------------- Resolution: Fixed > deadlock found in major compaction > ---------------------------------- > > Key: ACCUMULO-1110 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1110 > Project: Accumulo > Issue Type: Bug > Components: tserver > Environment: ten-node test cluster running random walk test > Reporter: Eric Newton > Assignee: Eric Newton > Priority: Critical > Labels: 15_qa_bug > Fix For: 1.5.0 > > > {noformat} > Found one Java-level deadlock: > ============================= > "ClientPool 131": > waiting to lock monitor 0x000000005e264210 (object 0x0000000765399a38, a org.apache.accumulo.server.tabletserver.Tablet), > which is held by "ClientPool 76" > "ClientPool 76": > waiting to lock monitor 0x0000000061ed94d0 (object 0x000000075a3a0478, a org.apache.accumulo.server.tabletserver.CompactionQueue), > which is held by "major compactor 3" > "major compactor 3": > waiting to lock monitor 0x00002aaac470b690 (object 0x0000000764d0d688, a org.apache.accumulo.server.tabletserver.Tablet), > which is held by "ClientPool 36" > "ClientPool 36": > waiting to lock monitor 0x0000000061ed94d0 (object 0x000000075a3a0478, a org.apache.accumulo.server.tabletserver.CompactionQueue), > which is held by "major compactor 3" > Java stack information for the threads listed above: > =================================================== > "ClientPool 131": > at org.apache.accumulo.server.tabletserver.Tablet.importMapFiles(Tablet.java:3666) > - waiting to lock <0x0000000765399a38> (a org.apache.accumulo.server.tabletserver.Tablet) > at org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.bulkImport(TabletServer.java:898) > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.accumulo.trace.instrument.thrift.TraceWrap$1.invoke(TraceWrap.java:63) > at $Proxy2.bulkImport(Unknown Source) > at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$bulkImport.getResult(TabletClientService.java:2153) > at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$bulkImport.getResult(TabletClientService.java:2137) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:156) > at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:478) > at org.apache.accumulo.server.util.TServerUtils$THsHaServer$Invocation.run(TServerUtils.java:208) > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47) > at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34) > at java.lang.Thread.run(Thread.java:662) > "ClientPool 76": > at org.apache.accumulo.server.tabletserver.CompactionQueue.offer(CompactionQueue.java:54) > - waiting to lock <0x000000075a3a0478> (a org.apache.accumulo.server.tabletserver.CompactionQueue) > at org.apache.accumulo.server.tabletserver.CompactionQueue.offer(CompactionQueue.java:28) > at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:653) > at org.apache.accumulo.trace.instrument.TraceExecutorService.execute(TraceExecutorService.java:39) > at org.apache.accumulo.server.tabletserver.TabletServerResourceManager.executeMajorCompaction(TabletServerResourceManager.java:717) > at org.apache.accumulo.server.tabletserver.TabletServerResourceManager$TabletResourceManager.executeMajorCompaction(TabletServerResourceManager.java:694) > at org.apache.accumulo.server.tabletserver.Tablet.initiateMajorCompaction(Tablet.java:2924) > - locked <0x0000000765399a38> (a org.apache.accumulo.server.tabletserver.Tablet) > at org.apache.accumulo.server.tabletserver.Tablet.importMapFiles(Tablet.java:368) > at org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.bulkImport(TabletServer.java:898) > at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.accumulo.trace.instrument.thrift.TraceWrap$1.invoke(TraceWrap.java:63) > at $Proxy2.bulkImport(Unknown Source) > at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$bulkImport.getResult(TabletClientService.java:2153) > at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$bulkImport.getResult(TabletClientService.java:2137) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:156) > at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:478) > at org.apache.accumulo.server.util.TServerUtils$THsHaServer$Invocation.run(TServerUtils.java:208) > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47) > at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34) > at java.lang.Thread.run(Thread.java:662) > "major compactor 3": > at org.apache.accumulo.server.tabletserver.Tablet$CompactionRunner.getNumFiles(Tablet.java:2895) > - waiting to lock <0x0000000764d0d688> (a org.apache.accumulo.server.tabletserver.Tablet) > at org.apache.accumulo.server.tabletserver.Tablet$CompactionRunner.compareTo(Tablet.java:2912) > at org.apache.accumulo.server.tabletserver.Tablet$CompactionRunner.compareTo(Tablet.java:2848) > at org.apache.accumulo.trace.instrument.TraceRunnable.compareTo(TraceRunnable.java:54) > at org.apache.accumulo.trace.instrument.TraceRunnable.compareTo(TraceRunnable.java:23) > at java.util.Collections.min(Collections.java:574) > at org.apache.accumulo.server.tabletserver.CompactionQueue.poll(CompactionQueue.java:38) > - locked <0x000000075a3a0478> (a org.apache.accumulo.server.tabletserver.CompactionQueue) > at org.apache.accumulo.server.tabletserver.CompactionQueue.take(CompactionQueue.java:78) > - locked <0x000000075a3a0478> (a org.apache.accumulo.server.tabletserver.CompactionQueue) > at org.apache.accumulo.server.tabletserver.CompactionQueue.take(CompactionQueue.java:28) > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) > at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47) > at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34) > at java.lang.Thread.run(Thread.java:662) > "ClientPool 36": > at org.apache.accumulo.server.tabletserver.CompactionQueue.offer(CompactionQueue.java:54) > - waiting to lock <0x000000075a3a0478> (a org.apache.accumulo.server.tabletserver.CompactionQueue) > at org.apache.accumulo.server.tabletserver.CompactionQueue.offer(CompactionQueue.java:28) > at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:653) > at org.apache.accumulo.trace.instrument.TraceExecutorService.execute(TraceExecutorService.java:39) > at org.apache.accumulo.server.tabletserver.TabletServerResourceManager.executeMajorCompaction(TabletServerResourceManager.java:717) > at org.apache.accumulo.server.tabletserver.TabletServerResourceManager$TabletResourceManager.executeMajorCompaction(TabletServerResourceManager.java:694) > at org.apache.accumulo.server.tabletserver.Tablet.initiateMajorCompaction(Tablet.java:2924) > - locked <0x0000000764d0d688> (a org.apache.accumulo.server.tabletserver.Tablet) > at org.apache.accumulo.server.tabletserver.Tablet.compactAll(Tablet.java:3871) > - locked <0x0000000764d0d688> (a org.apache.accumulo.server.tabletserver.Tablet) > at org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.compact(TabletServer.java:2048) > at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at org.apache.accumulo.trace.instrument.thrift.TraceWrap$1.invoke(TraceWrap.java:63) > at $Proxy2.compact(Unknown Source) > at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$compact.getResult(TabletClientService.java:2296) > at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor$compact.getResult(TabletClientService.java:2282) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:156) > at org.apache.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:478) > at org.apache.accumulo.server.util.TServerUtils$THsHaServer$Invocation.run(TServerUtils.java:208) > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47) > at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34) > at java.lang.Thread.run(Thread.java:662) > Found 1 deadlock. > {noformat} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira