cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Kania (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-11200) CompactionExecutor thread error brings down JVM
Date Sun, 21 Feb 2016 02:38:18 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-11200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jason Kania updated CASSANDRA-11200:
------------------------------------
    Description: 
When launching Cassandra 3.0.3, with java version "1.8.0_74", Cassandra writes the following
to the debug file before a segmentation fault occurs bringing down the JVM - the problem is
repeatable.

DEBUG [CompactionExecutor:1] 2016-02-20 18:26:16,892 CompactionTask.java:146 - Compacting
(56f677c0-d829-11e5-b23a-25dbd4d727f6) [/var/lib/cassandra/data/sensordata/periodicReading/ma-367-big-Data.db:level=0,
/var/lib/cassandra/data/sensordata/periodicReading/ma-368-big-Data.db:level=0, /var/lib/cassandra/data/sensordata/periodicReading/ma-371-big-Data.db:level=0,
/var/lib/cassandra/data/sensordata/periodicReading/ma-370-big-Data.db:level=0, /var/lib/cassandra/data/sensordata/periodicReading/ma-369-big-Data.db:level=0,
]

The JVM error that occurs is the following:

\#
\# A fatal error has been detected by the Java Runtime Environment:
\#
\#  SIGBUS (0x7) at pc=0x00007fa8a1052150, pid=12179, tid=140361951868672
\#
\# JRE version: Java(TM) SE Runtime Environment (8.0_74-b02) (build 1.8.0_74-b02)
\# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.74-b02 mixed mode linux-amd64 compressed
oops)
\# Problematic frame:
\# v  ~StubRoutines::jbyte_disjoint_arraycopy
\#
\# Core dump written. Default location: /tmp/core or core.12179
\#
\# If you would like to submit a bug report, please visit:
\#   http://bugreport.java.com/bugreport/crash.jsp
\#

---------------  T H R E A D  ---------------

Current thread (0x00007fa89c56ac20):  JavaThread "CompactionExecutor:1" daemon [_thread_in_Java,
id=12323, stack(0x00007fa89043f000,0x00007fa890480000)]

siginfo: si_signo: 7 (SIGBUS), si_code: 2 (BUS_ADRERR), si_addr: 0x00007fa838988002

Even if all of the files associated with "ma-[NNN]*" are removed, the JVM dies with the same
error after the next group of "ma-[NNN]*" are eventually written out and compacted.

Though this may be strictly a JVM problem, I have seen the issue in 8.0_65 and 8.0_74 and
I raise it in case this problem is due to JNI usage of an external compression library or
some direct memory usage.

I have a core dump if that is helpful to anyone.

Bug CASSANDRA-11201 may also be related although when the exception referenced in the bug
occurs, the JVM remains alive.

  was:
When launching Cassandra 3.0.3, with java version "1.8.0_74", Cassandra writes the following
to the debug file before a segmentation fault occurs bringing down the JVM - the problem is
repeatable.

DEBUG [CompactionExecutor:1] 2016-02-20 18:26:16,892 CompactionTask.java:146 - Compacting
(56f677c0-d829-11e5-b23a-25dbd4d727f6) [/var/lib/cassandra/data/sensordata/periodicReading/ma-367-big-Data.db:level=0,
/var/lib/cassandra/data/sensordata/periodicReading/ma-368-big-Data.db:level=0, /var/lib/cassandra/data/sensordata/periodicReading/ma-371-big-Data.db:level=0,
/var/lib/cassandra/data/sensordata/periodicReading/ma-370-big-Data.db:level=0, /var/lib/cassandra/data/sensordata/periodicReading/ma-369-big-Data.db:level=0,
]

The JVM error that occurs is the following:

\#
\# A fatal error has been detected by the Java Runtime Environment:
\#
\#  SIGBUS (0x7) at pc=0x00007fa8a1052150, pid=12179, tid=140361951868672
\#
\# JRE version: Java(TM) SE Runtime Environment (8.0_74-b02) (build 1.8.0_74-b02)
\# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.74-b02 mixed mode linux-amd64 compressed
oops)
\# Problematic frame:
\# v  ~StubRoutines::jbyte_disjoint_arraycopy
\#
\# Core dump written. Default location: /tmp/core or core.12179
\#
\# If you would like to submit a bug report, please visit:
\#   http://bugreport.java.com/bugreport/crash.jsp
\#

---------------  T H R E A D  ---------------

Current thread (0x00007fa89c56ac20):  JavaThread "CompactionExecutor:1" daemon [_thread_in_Java,
id=12323, stack(0x00007fa89043f000,0x00007fa890480000)]

siginfo: si_signo: 7 (SIGBUS), si_code: 2 (BUS_ADRERR), si_addr: 0x00007fa838988002

Even if all of the files associated with "ma-[NNN]*" are removed, the JVM dies with the same
error after the next group of "ma-[NNN]*" are eventually written out and compacted.

Though this may be strictly a JVM problem, I have seen the issue in 8.0_65 and 8.0_74 and
I raise it in case this problem is due to JNI usage of an external compression library or
some direct memory usage.

I have a core dump if that is helpful to anyone.

Bug CASSANDRA-11201 may also be related although when this exception occurs, the JVM remains
alive.


> CompactionExecutor thread error brings down JVM
> -----------------------------------------------
>
>                 Key: CASSANDRA-11200
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11200
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Compaction
>         Environment: debian jesse latest release, updated Feb. 20th
>            Reporter: Jason Kania
>            Priority: Critical
>
> When launching Cassandra 3.0.3, with java version "1.8.0_74", Cassandra writes the following
to the debug file before a segmentation fault occurs bringing down the JVM - the problem is
repeatable.
> DEBUG [CompactionExecutor:1] 2016-02-20 18:26:16,892 CompactionTask.java:146 - Compacting
(56f677c0-d829-11e5-b23a-25dbd4d727f6) [/var/lib/cassandra/data/sensordata/periodicReading/ma-367-big-Data.db:level=0,
/var/lib/cassandra/data/sensordata/periodicReading/ma-368-big-Data.db:level=0, /var/lib/cassandra/data/sensordata/periodicReading/ma-371-big-Data.db:level=0,
/var/lib/cassandra/data/sensordata/periodicReading/ma-370-big-Data.db:level=0, /var/lib/cassandra/data/sensordata/periodicReading/ma-369-big-Data.db:level=0,
]
> The JVM error that occurs is the following:
> \#
> \# A fatal error has been detected by the Java Runtime Environment:
> \#
> \#  SIGBUS (0x7) at pc=0x00007fa8a1052150, pid=12179, tid=140361951868672
> \#
> \# JRE version: Java(TM) SE Runtime Environment (8.0_74-b02) (build 1.8.0_74-b02)
> \# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.74-b02 mixed mode linux-amd64 compressed
oops)
> \# Problematic frame:
> \# v  ~StubRoutines::jbyte_disjoint_arraycopy
> \#
> \# Core dump written. Default location: /tmp/core or core.12179
> \#
> \# If you would like to submit a bug report, please visit:
> \#   http://bugreport.java.com/bugreport/crash.jsp
> \#
> ---------------  T H R E A D  ---------------
> Current thread (0x00007fa89c56ac20):  JavaThread "CompactionExecutor:1" daemon [_thread_in_Java,
id=12323, stack(0x00007fa89043f000,0x00007fa890480000)]
> siginfo: si_signo: 7 (SIGBUS), si_code: 2 (BUS_ADRERR), si_addr: 0x00007fa838988002
> Even if all of the files associated with "ma-[NNN]*" are removed, the JVM dies with the
same error after the next group of "ma-[NNN]*" are eventually written out and compacted.
> Though this may be strictly a JVM problem, I have seen the issue in 8.0_65 and 8.0_74
and I raise it in case this problem is due to JNI usage of an external compression library
or some direct memory usage.
> I have a core dump if that is helpful to anyone.
> Bug CASSANDRA-11201 may also be related although when the exception referenced in the
bug occurs, the JVM remains alive.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message