cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From William Boutin <william.bou...@ericsson.com>
Subject RE: Cassandra 2.2.6 Fails to Boot Up correctly - JNA Class
Date Thu, 20 Jul 2017 12:41:24 GMT
Thank you for your help.
We have been using jna-4.0.0.jar since using Cassandra 2.2(.6). Until last week, we had no
issues. Now, we are experiencing the exception that I identified. We only have jna-4.0.0.jar
loaded on our machines and the CLASSPATH that Cassandra builds only uses the jar from the
/usr/share/cassandra/lib directory. Jna-3.2.4.jar was only introduced when I was debugging
the original issue and will never be loaded on our machines. See the output of ps below.

Should I be looking elsewhere? Some have brought up that low memory on a JVM or LINUX machine
could cause jar loading issues. I have tried doubling -Xss256k to -Xss512k.

Thanks

Billy S. Boutin 
Office Phone No. (913) 241-5574 
Cell Phone No. (732) 213-1368 

496       3318     1  0 Jul19 ?        00:06:20 /usr/java/latest/bin/java -Dorg.xerial.snappy.tempdir=/var/tmp
-Djava.io.tmpdir=/var/tmp -ea -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar -XX:+CMSClassUnloadingEnabled
-XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms1968M -Xmx1968M -Xmn200M -XX:+HeapDumpOnOutOfMemoryError
-Xss256k -XX:StringTableSize=1000003 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled
-XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=1 -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
-XX:+UseTLAB -XX:+PerfDisableSharedMem -XX:CompileCommandFile=/etc/cassandra/conf/hotspot_compiler
-XX:CMSWaitDuration=10000 -XX:+CMSParallelInitialMarkEnabled -XX:+CMSEdenChunksRecordAlways
-XX:CMSWaitDuration=10000 -XX:+UseCondCardMark -XX:+PrintGCDetails -XX:+PrintGCDateStamps
-XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -XX:+PrintPromotionFailure
-Xloggc:/usr/share/cassandra/logs/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10
-XX:GCLogFileSize=10M -Djava.net.preferIPv4Stack=true -Dcassandra.jmx.local.port=7199 -XX:+DisableExplicitGC
-Djava.library.path=/usr/share/cassandra/lib/sigar-bin -Dlogback.configurationFile=logback.xml
-Dcassandra.logdir=/usr/share/cassandra/logs -Dcassandra.storagedir= -Dcassandra-pidfile=/var/run/cassandra/cassandra.pid
-cp /etc/cassandra/conf:/usr/share/cassandra/lib/airline-0.6.jar:/usr/share/cassandra/lib/antlr-runtime-3.5.2.jar:/usr/share/cassandra/lib/apache-cassandra-clientutil-2.2.6-E002.jar:/usr/share/cassandra/lib/cassandra-driver-core-2.2.0-rc2-SNAPSHOT-20150617-shaded.jar:/usr/share/cassandra/lib/commons-cli-1.1.jar:/usr/share/cassandra/lib/commons-codec-1.2.jar:/usr/share/cassandra/lib/commons-lang3-3.1.jar:/usr/share/cassandra/lib/commons-math3-3.2.jar:/usr/share
/cassandra/lib/compress-lzf-0.8.4.jar:/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.4.jar:/usr/share/cassandra/lib/crc32ex-0.1.1.jar:/usr/share/cassandra/lib/disruptor-3.0.1.jar:/usr/share/cassandra/lib/ecj-4.4.2.jar:/usr/share/cassandra/lib/guava-16.0.jar:/usr/share/cassandra/lib/high-scale-lib-1.0.6.jar:/usr/share/cassandra/lib/jackson-core-asl-1.9.2.jar:/usr/share/cassandra/lib/jackson-mapper-asl-1.9.2.jar:/usr/share/cassandra/lib/jamm-0.3.0.jar:/usr/share/cassandra/lib/javax.inject.jar:/usr/share/cassandra/lib/jbcrypt-0.3m.jar:/usr/share/cassandra/lib/jcl-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/jna-4.0.0.jar:/usr/share/cassandra/lib/joda-time-2.4.jar:/usr/share/cassandra/lib/json-simple-1.1.jar:/usr/share/cassandra/lib/libthrift-0.9.2.jar:/usr/share/cassandra/lib/log4j-over-slf4j-1.7.7.jar:/usr/share/cassandra/lib/logback-classic-1.1.3.jar:/usr/share/cassandra/lib/logback-core-1.1.3.jar:/usr/share/cassandra/lib/lz4-1.3.0.jar:/usr/share/cassandra/lib/metrics-core-3.1.0.jar:/usr/share/cassandra/lib/metrics-logback-3.1.0.jar:/usr/share/cassandra/lib/netty-all-4.0.23.Final.jar:/usr/share/cassandra/lib/ohc-core-0.3.4.jar:/usr/share/cassandra/lib/ohc-core-j8-0.3.4.jar:/usr/share/cassandra/lib/reporter-config3-3.0.0.jar:/usr/share/cassandra/lib/reporter-config-base-3.0.0.jar:/usr/share/cassandra/lib/sigar-1.6.4.jar:/usr/share/cassandra/lib/slf4j-api-1.7.7.jar:/usr/share/cassandra/lib/snakeyaml-1.11.jar:/usr/share/cassandra/lib/snappy-java-1.1.1.7.jar:/usr/share/cassandra/lib/ST4-4.0.8.jar:/usr/share/cassandra/lib/stream-2.5.2.jar:/usr/share/cassandra/lib/super-c
sv-2.1.0.jar:/usr/share/cassandra/lib/thrift-server-0.3.7.jar:/usr/share/cassandra/apache-cassandra-2.2.6-E002.jar:/usr/share/cassandra/apache-cassandra-thrift-2.2.6-E002.jar:/usr/share/cassandra/stress.jar:
org.apache.cassandra.service.CassandraDaemon

-----Original Message-----
From: Jeff Jirsa [mailto:jjirsa@apache.org] 
Sent: Wednesday, July 19, 2017 11:47 PM
To: user@cassandra.apache.org
Subject: Re: Cassandra 2.2.6 Fails to Boot Up correctly - JNA Class



On 2017-07-19 10:41 (-0700), William Boutin <william.boutin@ericsson.com> wrote: 
> We are running apache-cassandra-2.2.6 for months with no JNA startup issues.
> Recently, we have updated some of our cassandra machines and we ran into a Cassandra
startup issue with JNA. See the stack trace 1 below.
> 
> Question 1: Our Cassandra package contains jna-4.0.0.jar in the /usr/share/cassandra/lib/
directory. Does apache-cassandra-2.2.6 specifically package A-C* with the jna-4.0.0.jar ?
> 

Yes, we explicitly bundle jna-4.0.0.jar for 2.2.6 

This is unchanged in 2.2 since https://github.com/apache/cassandra/commit/2ba394676dc673b6d66a07247dccd122b64b0578
( Jan 2014, when it was cassandra 2.1 beta).

Is the jar missing? Did another version of jna get installed? If a system package got installed,
you may have multiple versions in the classpath. 

> I tried to circumvent the exception by yum installing  jna-3.2.4.jar for RHEL 6.4 and
adding it to the /usr/share/cassandra/lib/ directory. This got me past the JNA mlockall failure
with "INFO  16:56:47 JNA mlockall successful" but when the MemtableFlushWriter requested memory
allocation, I now get the STACK TRACE 2 error. "nodetool status" still reports the node as
incomplete.
> 

Installing 3.2.4 is probably not going to work (as hinted in my earlier paragraph); you almost
certainly want 4.0 or newer, and you should probably want exactly one jna jar in the classpath
at any given time.
 


- Jeff

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org


Mime
View raw message