Return-Path: X-Original-To: apmail-accumulo-notifications-archive@minotaur.apache.org Delivered-To: apmail-accumulo-notifications-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C0BC4107A3 for ; Tue, 18 Nov 2014 20:34:35 +0000 (UTC) Received: (qmail 78013 invoked by uid 500); 18 Nov 2014 20:34:35 -0000 Delivered-To: apmail-accumulo-notifications-archive@accumulo.apache.org Received: (qmail 77981 invoked by uid 500); 18 Nov 2014 20:34:35 -0000 Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: jira@apache.org Delivered-To: mailing list notifications@accumulo.apache.org Received: (qmail 77968 invoked by uid 99); 18 Nov 2014 20:34:35 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 Nov 2014 20:34:35 +0000 Date: Tue, 18 Nov 2014 20:34:35 +0000 (UTC) From: "John Vines (JIRA)" To: notifications@accumulo.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Reopened] (ACCUMULO-1628) NPE on deep copied dumped memory iterator MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/ACCUMULO-1628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Vines reopened ACCUMULO-1628: ---------------------------------- I think this has recurred in 1.6.0 in a different with the patch you included. {code}2014-11-14 22:08:12,745 [tserver.InMemoryMap] ERROR: Failed to create mem dump file java.io.EOFException at java.io.DataInputStream.readByte(DataInputStream.java:267) at org.apache.accumulo.core.file.rfile.RelativeKey.fastSkip(RelativeKey.java:314) at org.apache.accumulo.core.file.rfile.RFile$LocalityGroupReader._seek(RFile.java:748) at org.apache.accumulo.core.file.rfile.RFile$LocalityGroupReader.seek(RFile.java:607) at org.apache.accumulo.core.iterators.system.LocalityGroupIterator.seek(LocalityGroupIterator.java:142) at org.apache.accumulo.core.file.rfile.RFile$Reader.seek(RFile.java:979) at org.apache.accumulo.core.iterators.WrappingIterator.seek(WrappingIterator.java:101) at org.apache.accumulo.tserver.MemKeyConversionIterator.seek(InMemoryMap.java:168) at org.apache.accumulo.core.iterators.system.SourceSwitchingIterator._switchNow(SourceSwitchingIterator.java:171) at org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.switchNow(SourceSwitchingIterator.java:179) at org.apache.accumulo.tserver.InMemoryMap$MemoryIterator.switchNow(InMemoryMap.java:647) at org.apache.accumulo.tserver.InMemoryMap$MemoryIterator.access$900(InMemoryMap.java:601) at org.apache.accumulo.tserver.InMemoryMap.delete(InMemoryMap.java:746) at org.apache.accumulo.tserver.Tablet$TabletMemory.finalizeMinC(Tablet.java:327) at org.apache.accumulo.tserver.Tablet.minorCompact(Tablet.java:2068) at org.apache.accumulo.tserver.Tablet.access$4300(Tablet.java:170) at org.apache.accumulo.tserver.Tablet$MinorCompactionTask.run(Tablet.java:2134) at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34) at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47) at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34) at java.lang.Thread.run(Thread.java:744){code} We have an installation that is large enough with high ingest and query that is triggering it regularly. I'm currently working on acquiring the datanode logs on the other end, but the footprint and conditions for this are near identical to the original cases we were seeing the old stack traces occur in. > NPE on deep copied dumped memory iterator > ----------------------------------------- > > Key: ACCUMULO-1628 > URL: https://issues.apache.org/jira/browse/ACCUMULO-1628 > Project: Accumulo > Issue Type: Bug > Affects Versions: 1.4.0, 1.5.0 > Reporter: Keith Turner > Assignee: Keith Turner > Priority: Blocker > Fix For: 1.5.2, 1.6.1, 1.7.0 > > Attachments: ACCUMULO-1628-1.5.2-SNAPSHOT-1.patch, ACCUMULO-1628-1.5.2-SNAPSHOT-2.patch, ACCUMULO-1628-1.5.2-SNAPSHOT-3.patch, ACCUMULO-1628-1.patch, ACCUMULO-1628-SET_INTR.patch, bpopp-ACCUMULO-1628.log > > Time Spent: 1h > Remaining Estimate: 0h > > Accumulo may dump memory while a scan is running and transparently switch iterators from memory to file. If an iterator calls deepcopy after this happens, then seek will fail on the deep copy. > {noformat} > java.lang.NullPointerException > at org.apache.accumulo.server.tabletserver.InMemoryMap$MemoryDataSource.iterator(InMemoryMap.java:349) > at org.apache.accumulo.core.iterators.system.SourceSwitchingIterator.seek(SourceSwitchingIterator.java:163) > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)