cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Jirsa (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-11847) Cassandra dies on a specific node in a multi-DC environment
Date Thu, 19 May 2016 18:21:12 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-11847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15291694#comment-15291694
] 

Jeff Jirsa commented on CASSANDRA-11847:
----------------------------------------

Your crashes are all over cassandra (commitlog, mutation, compaction) - the most likely cause
is bad hardware (bad memory, for example).

Physical hardware / home grown VM or public cloud? ECC RAM? 



> Cassandra dies on a specific node in a multi-DC environment
> -----------------------------------------------------------
>
>                 Key: CASSANDRA-11847
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11847
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Compaction, Core
>         Environment: Cassandra 2.0.11, JDK build 1.7.0_79-b15
>            Reporter: Rajesh Babu
>         Attachments: java_error19030.log, java_error2912.log, java_error4571.log, java_error7539.log,
java_error9552.log
>
>
> We've a customer who runs a 16 node 2 DC (8 nodes each) environment where Cassandra pid
dies randomly but on a specific node.
> Whenever Cassandra dies, admin has to manually restart Cassandra only on that node.
> I tried upgrading their environment from java 1.7 (patch 60) to java 1.7 (patch 79) but
it still seems to be an issue. 
> Is this a known hardware related bug or should is this issue fixed in later Cassandra
versions? 
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00007f4542d5a27f, pid=19030, tid=139933154096896
> #
> # JRE version: Java(TM) SE Runtime Environment (7.0_79-b15) (build 1.7.0_79-b15)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.79-b02 mixed mode linux-amd64 compressed
oops)
> # Problematic frame:
> # C  [libjava.so+0xe027f]  _fini+0xbd5f7
> #
> # Core dump written. Default location: /tmp/core or core.19030
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> #
> ---------------  T H R E A D  ---------------
> Current thread (0x00007f453c89f000):  JavaThread "COMMIT-LOG-WRITER" [_thread_in_vm,
id=19115, stack(0x00007f44b9ed3000,0x00007f44b9f14000)]
> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=2 (SEGV_ACCERR), si_addr=0x00007f4542d5a27f
> Registers:
> RAX=0x0000000000000000, RBX=0x00007f453c564ad0, RCX=0x0000000000000001, RDX=0x0000000000000020
> RSP=0x00007f44b9f125a0, RBP=0x00007f44b9f125b0, RSI=0x0000000000000000, RDI=0x0000000000000001
> R8 =0x00007f453c564ad8, R9 =0x0000000000004aab, R10=0x00007f453917a52c, R11=0x00000006fae57068
> R12=0x00007f453c564ad8, R13=0x00007f44b9f125d0, R14=0x0000000000000000, R15=0x00007f453c89f000
> RIP=0x00007f4542d5a27f, EFLAGS=0x0000000000010246, CSGSFS=0x0000000000000033, ERR=0x0000000000000014
>   TRAPNO=0x000000000000000e
> -------------------------------------
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00007f28e08787a4, pid=2912, tid=139798767699712
> #
> # JRE version: Java(TM) SE Runtime Environment (7.0_79-b15) (build 1.7.0_79-b15)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.79-b02 mixed mode linux-amd64 compressed
oops)
> # Problematic frame:
> # C  0x00007f28e08787a4
> #
> # Core dump written. Default location: /tmp/core or core.2912
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> #
> ---------------  T H R E A D  ---------------
> Current thread (0x00007f2640008000):  JavaThread "ValidationExecutor:15" daemon [_thread_in_Java,
id=7393, stack(0x00007f256fdf8000,0x00007f256fe39000)]
> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=2 (SEGV_ACCERR), si_addr=0x00007f28e08787a4
> Registers:
> RAX=0x00000000ffffffff, RBX=0x000000003f8bb878, RCX=0x00000000c77040d6, RDX=0x00000000c770409a
> RSP=0x00007f256fe37430, RBP=0x000000063b820710, RSI=0x000000063b820530, RDI=0x00000000ffffffff
> R8 =0x000000003f8bb888, R9 =0x0000000000000000, R10=0x000000003f8bb888, R11=0x000000003f8bb878
> R12=0x0000000000000000, R13=0x000000063b820530, R14=0x000000000000000b, R15=0x00007f2640008000
> RIP=0x00007f28e08787a4, EFLAGS=0x0000000000010246, CSGSFS=0x0000000000000033, ERR=0x0000000000000015
>   TRAPNO=0x000000000000000e



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message