cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Ravn (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-13503) Segfault during compaction
Date Mon, 08 May 2017 11:18:04 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-13503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16000615#comment-16000615
] 

James Ravn commented on CASSANDRA-13503:
----------------------------------------

{noformat}
Cassandra 2.1.17

java -version:
java version "1.8.0_112"
Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)

uname -a (Ubuntu 16.04.2):
Linux ip-10-50-194-251 4.8.0-51-generic #54~16.04.1-Ubuntu SMP Wed Apr 26 16:00:28 UTC 2017
x86_64 x86_64 x86_64 GNU/Linux

lstopo:
Machine (16GB total)
  NUMANode L#0 (P#0 7998MB) + Package L#0 + L3 L#0 (45MB) + L2 L#0 (256KB) + L1d L#0 (32KB)
+ L1i L#0 (32KB) + Core L#0
    PU L#0 (P#0)
    PU L#1 (P#1)
  NUMANode L#1 (P#1 8047MB) + Package L#1 + L3 L#1 (45MB) + L2 L#1 (256KB) + L1d L#1 (32KB)
+ L1i L#1 (32KB) + Core L#1
    PU L#2 (P#2)
    PU L#3 (P#3)
{noformat}

> Segfault during compaction
> --------------------------
>
>                 Key: CASSANDRA-13503
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13503
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Compaction
>         Environment: {noformat}
> Cassandra 2.1.17
> java -version:
> java version "1.8.0_112"
> Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
> Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)
> uname -a (Ubuntu 16.04.2):
> Linux ip-10-50-194-251 4.8.0-51-generic #54~16.04.1-Ubuntu SMP Wed Apr 26 16:00:28 UTC
2017 x86_64 x86_64 x86_64 GNU/Linux
> lstopo:
> Machine (16GB total)
>   NUMANode L#0 (P#0 7998MB) + Package L#0 + L3 L#0 (45MB) + L2 L#0 (256KB) + L1d L#0
(32KB) + L1i L#0 (32KB) + Core L#0
>     PU L#0 (P#0)
>     PU L#1 (P#1)
>   NUMANode L#1 (P#1 8047MB) + Package L#1 + L3 L#1 (45MB) + L2 L#1 (256KB) + L1d L#1
(32KB) + L1i L#1 (32KB) + Core L#1
>     PU L#2 (P#2)
>     PU L#3 (P#3)
> {noformat}
>            Reporter: James Ravn
>         Attachments: hs_err_pid29774.log
>
>
> One of our cassandra nodes segfaulted. I've attached the hs_err.log. It looks like it
was loading sstables from disk:
> {noformat}
> Current thread (0x00007f3df4e7b930):  JavaThread "CompactionExecutor:2" daemon [_thread_in_Java,
id=30023, stack(0x00007f3dce922000,0x00007f3dce963000)]
> siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0xffffffffbd49a8b6
> Stack: [0x00007f3dce922000,0x00007f3dce963000],  sp=0x00007f3dce961430,  free space=253k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
> J 11168 C2 org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext()Ljava/lang/Object;
(48 bytes) @ 0x00007f3de6097df4 [0x00007f3de6097cc0+0x134]
> J 2042 C2 com.google.common.collect.AbstractIterator.hasNext()Z (65 bytes) @ 0x00007f3de560fbc0
[0x00007f3de560fb00+0xc0]
> J 14662 C2 org.apache.cassandra.db.ColumnIndex$Builder.buildForCompaction(Ljava/util/Iterator;)Lorg/apache/cassandra/db/ColumnIndex;
(36 bytes) @ 0x00007f3de7143a08 [0x00007f3de7143440+0x5c8]
> J 13756 C2 org.apache.cassandra.io.sstable.SSTableRewriter.append(Lorg/apache/cassandra/db/compaction/AbstractCompactedRow;)Lorg/apache/cassandra/db/RowIndexEntry;
(119 bytes) @ 0x00007f3de6a98b88 [0x00007f3de6a985c0+0x5c8]
> J 14627 C2 org.apache.cassandra.db.compaction.CompactionTask.runMayThrow()V (1622 bytes)
@ 0x00007f3de710eed4 [0x00007f3de710daa0+0x1434]
> J 18294 C1 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(Lorg/apache/cassandra/db/compaction/CompactionManager$CompactionExecutorStatsCollector;)I
(19 bytes) @ 0x00007f3de7981aa4 [0x00007f3de79818c0+0x1e4]
> J 18503 C1 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(Lorg/apache/cassandra/db/compaction/CompactionManager$CompactionExecutorStatsCollector;)I
(39 bytes) @ 0x00007f3de7a4cf2c [0x00007f3de7a4ce20+0x10c]
> J 17908 C2 org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run()V
(182 bytes) @ 0x00007f3de789e0a4 [0x00007f3de789aca0+0x3404]
> {noformat}
> No errors in kernel logs, and no other noticeable issues on the node.
> We're using {{offheap_objects}}, could that be related? Cassandra logs show the usual
memtable flushing.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message