Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 8 May 2017 11:18:04 +0000 (UTC)
From: "James Ravn (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.13069957.1494241381000.150834.1494242284083@Atlassian.JIRA>
In-Reply-To: <JIRA.13069957.1494241381000@Atlassian.JIRA>
References: <JIRA.13069957.1494241381000@Atlassian.JIRA> <JIRA.13069957.1494241381615@jira-lw-us.apache.org>
Subject: [jira] [Updated] (CASSANDRA-13503) Segfault during compaction
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 08 May 2017 11:18:08 -0000


     [ https://issues.apache.org/jira/browse/CASSANDRA-13503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Ravn updated CASSANDRA-13503:
-----------------------------------
    Environment: Ubuntu 16.04.2  (was: {noformat}
Cassandra 2.1.17

java -version:
java version "1.8.0_112"
Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)

uname -a (Ubuntu 16.04.2):
Linux ip-10-50-194-251 4.8.0-51-generic #54~16.04.1-Ubuntu SMP Wed Apr 26 16:00:28 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

lstopo:
Machine (16GB total)
  NUMANode L#0 (P#0 7998MB) + Package L#0 + L3 L#0 (45MB) + L2 L#0 (256KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0
    PU L#0 (P#0)
    PU L#1 (P#1)
  NUMANode L#1 (P#1 8047MB) + Package L#1 + L3 L#1 (45MB) + L2 L#1 (256KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1
    PU L#2 (P#2)
    PU L#3 (P#3)
{noformat})

> Segfault during compaction
> --------------------------
>
>                 Key: CASSANDRA-13503
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13503
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Compaction
>         Environment: Ubuntu 16.04.2
>            Reporter: James Ravn
>         Attachments: hs_err_pid29774.log
>
>
> One of our cassandra nodes segfaulted. I've attached the hs_err.log. It looks like it was loading sstables from disk:
> {noformat}
> Current thread (0x00007f3df4e7b930):  JavaThread "CompactionExecutor:2" daemon [_thread_in_Java, id=30023, stack(0x00007f3dce922000,0x00007f3dce963000)]
> siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0xffffffffbd49a8b6
> Stack: [0x00007f3dce922000,0x00007f3dce963000],  sp=0x00007f3dce961430,  free space=253k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> J 11168 C2 org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext()Ljava/lang/Object; (48 bytes) @ 0x00007f3de6097df4 [0x00007f3de6097cc0+0x134]
> J 2042 C2 com.google.common.collect.AbstractIterator.hasNext()Z (65 bytes) @ 0x00007f3de560fbc0 [0x00007f3de560fb00+0xc0]
> J 14662 C2 org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(Ljava/util/Iterator;)Lorg/apache/cassandra/db/ColumnIndex; (36 bytes) @ 0x00007f3de7143a08 [0x00007f3de7143440+0x5c8]
> J 13756 C2 org.apache.cassandra.io.sstable.SSTableRewriter.append(Lorg/apache/cassandra/db/compaction/AbstractCompactedRow;)Lorg/apache/cassandra/db/RowIndexEntry; (119 bytes) @ 0x00007f3de6a98b88 [0x00007f3de6a985c0+0x5c8]
> J 14627 C2 org.apache.cassandra.db.compaction.CompactionTask.runMayThrow()V (1622 bytes) @ 0x00007f3de710eed4 [0x00007f3de710daa0+0x1434]
> J 18294 C1 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(Lorg/apache/cassandra/db/compaction/CompactionManager$CompactionExecutorStatsCollector;)I (19 bytes) @ 0x00007f3de7981aa4 [0x00007f3de79818c0+0x1e4]
> J 18503 C1 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(Lorg/apache/cassandra/db/compaction/CompactionManager$CompactionExecutorStatsCollector;)I (39 bytes) @ 0x00007f3de7a4cf2c [0x00007f3de7a4ce20+0x10c]
> J 17908 C2 org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run()V (182 bytes) @ 0x00007f3de789e0a4 [0x00007f3de789aca0+0x3404]
> {noformat}
> No errors in kernel logs, and no other noticeable issues on the node.
> We're using {{offheap_objects}}, could that be related? Cassandra logs show the usual memtable flushing.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org