incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Stepura" <mikhail.step...@outlook.com>
Subject Re: Cassandra shows java exceptions
Date Thu, 28 Nov 2013 01:59:10 GMT
What’s the value of “max_hint_window_in_ms” in your cassandra.yaml?

-M


"Xavier Fustero" <xavier@rightscale.com> wrote in message news:CAH7ZuUsuh7S_9vvJAXg75fo5xd2rPwy6Rgpw_UBxMwBWd4_X4g@mail.gmail.com...
Hi all,

I am installing a cassandra 1.2 on Ubuntu. I followed the Debian/Ubuntu guidelines but even
following the procedure to get rid of openJDK it was always there. I read the packages a build
using openJDK so I am not sure who really get rid of it. 

Anyway, I finally decided to install it from apache-cassandra tar file. However, after my
instance is booted with all configuration, starting it on foreground shows me the error below.
I got the same before using deb package but I thought it was related to the openJDK but it
seems it is not as I am using 

# java -version
java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b04)
Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)

I also updated the mx4j-tools.jar in case it was related. The system.log doesn't seem to show
any error thoug. I need these MBeans working as I am monitoring Cassandra with some collectd
plugins accessing to them.

Please let me know if anyone understand what is wrong here. I pasted below all the logs I
am talking about.

Thanks a lot,
Xavi


root@xaviring1:~# /usr/share/cassandra/bin/cassandra -f

xss =  -XX:HeapDumpPath=/mnt/cassandra -XX:+CMSIncrementalPacing -XX:+CMSIncrementalMode -javaagent:/usr/share/cassandra/bin/../lib/jamm-0.2.5.jar
-XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms1G -Xmx1G -Xmn100M -XX:+HeapDumpOnOutOfMemoryError
-Xss256k
com.rightscale.rmiregistry.port:8081
Getting the platform's MBean Server
Connection URL: service:jmx:rmi://localhost:8081/jndi/rmi://localhost:8081/jmxrmi
Creating RMI connector server
HttpAdaptor version 3.0.2 started on port 8082
Exception during http request
javax.management.RuntimeMBeanException: java.lang.NullPointerException
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:856)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:869)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:670)
at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:638)
at mx4j.tools.adaptor.http.MBeanCommandProcessor.createMBeanElement(MBeanCommandProcessor.java:119)
at mx4j.tools.adaptor.http.MBeanCommandProcessor.executeRequest(MBeanCommandProcessor.java:56)
at mx4j.tools.adaptor.http.HttpAdaptor$HttpClient.run(HttpAdaptor.java:980)
Caused by: java.lang.NullPointerException
at org.apache.cassandra.config.DatabaseDescriptor.getMaxHintWindow(DatabaseDescriptor.java:1157)
at org.apache.cassandra.service.StorageProxy.getMaxHintWindow(StorageProxy.java:1506)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:65)
at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:216)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:666)
... 4 more


Here is my system.log:

Nov 27 16:41:05 xaviring1 cassandra: Logging initialized  
Nov 27 16:41:05 xaviring1 cassandra: JVM vendor/version: Java HotSpot(TM) 64-Bit Server VM/1.6.0_31
 
Nov 27 16:41:05 xaviring1 cassandra: Your JVM is out of date. Please upgrade to the newest
Oracle Java 6.  
Nov 27 16:41:05 xaviring1 cassandra: Heap size: 1063256064/1063256064  
Nov 27 16:41:05 xaviring1 cassandra: Classpath: /usr/share/cassandra/bin/../conf:/usr/share/cassandra/bin/../build/classes/main:/usr/share/cassandra/bin/../build/classes/thrift:/usr/share/cassandra/bin/../lib/CustomAgent.jar:/usr/share/cassandra/bin/../lib/antlr-3.2.jar:/usr/share/cassandra/bin/../lib/apache-cassandra-1.2.12.jar:/usr/share/cassandra/bin/../lib/apache-cassandra-clientutil-1.2.12.jar:/usr/share/cassandra/bin/../lib/apache-cassandra-thrift-1.2.12.jar:/usr/share/cassandra/bin/../lib/apache-cassandra.jar:/usr/share/cassandra/bin/../lib...
Nov 27 16:41:06 xaviring1 cassandra: JNA mlockall successful  
Nov 27 16:41:06 xaviring1 cassandra: Loading settings from file:/etc/cassandra/cassandra.yaml
 
Nov 27 16:41:07 xaviring1 cassandra: Data files directories: [/mnt/cassandra/data]  
Nov 27 16:41:07 xaviring1 cassandra: Commit log directory: /mnt/cassandra/commitlog  
Nov 27 16:41:07 xaviring1 cassandra: DiskAccessMode 'auto' determined to be mmap, indexAccessMode
is mmap  
Nov 27 16:41:07 xaviring1 cassandra: disk_failure_policy is stop  
Nov 27 16:41:07 xaviring1 cassandra: Global memtable threshold is enabled at 338MB  
Nov 27 16:41:07 xaviring1 cassandra: Not using multi-threaded compaction  
Nov 27 16:41:07 xaviring1 cassandra: Please rename encryption_options as server_encryption_options
in the yaml  
Nov 27 16:41:08 xaviring1 cassandra: Initializing key cache with capacity of 100 MBs.  
Nov 27 16:41:08 xaviring1 cassandra: Scheduling key cache save to each 14400 seconds (going
to save all keys).  
Nov 27 16:41:08 xaviring1 cassandra: Initializing row cache with capacity of 100 MBs and provider
org.apache.cassandra.cache.SerializingCacheProvider  
Nov 27 16:41:08 xaviring1 cassandra: Scheduling row cache save to each 0 seconds (going to
save all keys).  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.batchlog  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.peer_events  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.hints  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.Schema  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.schema_keyspaces  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ic-1
(220 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ic-1
(220 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ic-2
(220 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ic-2
(220 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.range_xfers  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.HintsColumnFamily  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.schema_columnfamilies  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ic-2
(4593 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ic-2
(4593 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ic-1
(4593 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ic-1
(4593 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.NodeIdInfo  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.schema_columns  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/schema_columns/system-schema_columns-ic-1
(4164 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/schema_columns/system-schema_columns-ic-1
(4164 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/schema_columns/system-schema_columns-ic-2
(4177 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/schema_columns/system-schema_columns-ic-2
(4177 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.IndexInfo  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.Migrations  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.peers  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.local  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/local/system-local-ic-11
(5746 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/local/system-local-ic-11
(5746 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/local/system-local-ic-12
(120 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/local/system-local-ic-12
(120 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/local/system-local-ic-13
(5421 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Opening /mnt/cassandra/data/system/local/system-local-ic-13
(5421 bytes)  
Nov 27 16:41:08 xaviring1 cassandra: Initializing system.LocationInfo  
Nov 27 16:41:09 xaviring1 cassandra: Initializing system_traces.sessions  
Nov 27 16:41:09 xaviring1 cassandra: Initializing system_traces.events  
Nov 27 16:41:09 xaviring1 cassandra: completed pre-loading (3 keys) key cache.  
Nov 27 16:41:09 xaviring1 cassandra: Replaying /mnt/cassandra/commitlog/CommitLog-2-1385569023050.log,
/mnt/cassandra/commitlog/CommitLog-2-1385569023051.log, /mnt/cassandra/commitlog/CommitLog-2-1385569023052.log,
/mnt/cassandra/commitlog/CommitLog-2-1385569023053.log  
Nov 27 16:41:09 xaviring1 cassandra: Replaying /mnt/cassandra/commitlog/CommitLog-2-1385569023050.log
 
Nov 27 16:41:09 xaviring1 cassandra: Finished reading /mnt/cassandra/commitlog/CommitLog-2-1385569023050.log
 
Nov 27 16:41:09 xaviring1 cassandra: Replaying /mnt/cassandra/commitlog/CommitLog-2-1385569023051.log
 
Nov 27 16:41:09 xaviring1 cassandra: Finished reading /mnt/cassandra/commitlog/CommitLog-2-1385569023051.log
 
Nov 27 16:41:09 xaviring1 cassandra: Replaying /mnt/cassandra/commitlog/CommitLog-2-1385569023052.log
 
Nov 27 16:41:09 xaviring1 cassandra: Finished reading /mnt/cassandra/commitlog/CommitLog-2-1385569023052.log
 
Nov 27 16:41:09 xaviring1 cassandra: Replaying /mnt/cassandra/commitlog/CommitLog-2-1385569023053.log
 
Nov 27 16:41:09 xaviring1 cassandra: Finished reading /mnt/cassandra/commitlog/CommitLog-2-1385569023053.log
 
Nov 27 16:41:09 xaviring1 cassandra: Enqueuing flush of Memtable-local@420824846(52/520 serialized/live
bytes, 2 ops)  
Nov 27 16:41:09 xaviring1 cassandra: Writing Memtable-local@420824846(52/520 serialized/live
bytes, 2 ops)  
Nov 27 16:41:09 xaviring1 cassandra: Enqueuing flush of Memtable-schema_keyspaces@1503990546(251/2510
serialized/live bytes, 8 ops)  
Nov 27 16:41:09 xaviring1 cassandra: Enqueuing flush of Memtable-schema_columns@1686581432(23741/237410
serialized/live bytes, 386 ops)  
Nov 27 16:41:09 xaviring1 cassandra: Enqueuing flush of Memtable-schema_columnfamilies@1737470871(22187/221870
serialized/live bytes, 369 ops)  
Nov 27 16:41:10 xaviring1 cassandra: Completed flushing /mnt/cassandra/data/system/local/system-local-ic-14-Data.db
(84 bytes) for commitlog position ReplayPosition(segmentId=1385570469088, position=142)  
Nov 27 16:41:10 xaviring1 cassandra: Writing Memtable-schema_keyspaces@1503990546(251/2510
serialized/live bytes, 8 ops)  
Nov 27 16:41:10 xaviring1 cassandra: Completed flushing /mnt/cassandra/data/system/schema_keyspaces/system-schema_keyspaces-ic-3-Data.db
(218 bytes) for commitlog position ReplayPosition(segmentId=1385570469088, position=142) 

Nov 27 16:41:10 xaviring1 cassandra: Writing Memtable-schema_columns@1686581432(23741/237410
serialized/live bytes, 386 ops)  
Nov 27 16:41:10 xaviring1 cassandra: Completed flushing /mnt/cassandra/data/system/schema_columns/system-schema_columns-ic-3-Data.db
(4164 bytes) for commitlog position ReplayPosition(segmentId=1385570469088, position=142)
 
Nov 27 16:41:10 xaviring1 cassandra: Writing Memtable-schema_columnfamilies@1737470871(22187/221870
serialized/live bytes, 369 ops)  
Nov 27 16:41:10 xaviring1 cassandra: Completed flushing /mnt/cassandra/data/system/schema_columnfamilies/system-schema_columnfamilies-ic-3-Data.db
(4593 bytes) for commitlog position ReplayPosition(segmentId=1385570469088, position=142)
 
Nov 27 16:41:10 xaviring1 cassandra: Log replay complete, 12 replayed mutations  
Nov 27 16:41:10 xaviring1 cassandra: Cassandra version: 1.2.12  
Nov 27 16:41:10 xaviring1 cassandra: Thrift API version: 19.36.1  
Nov 27 16:41:10 xaviring1 cassandra: CQL supported versions: 2.0.0,3.0.5 (default: 3.0.5)
 
Nov 27 16:41:10 xaviring1 cassandra: Loading persisted ring state  
Nov 27 16:41:10 xaviring1 cassandra: Starting up server gossip  
Nov 27 16:41:11 xaviring1 cassandra: Enqueuing flush of Memtable-local@413461956(249/2490
serialized/live bytes, 9 ops)  
Nov 27 16:41:11 xaviring1 cassandra: Writing Memtable-local@413461956(249/2490 serialized/live
bytes, 9 ops)  
Nov 27 16:41:11 xaviring1 cassandra: Completed flushing /mnt/cassandra/data/system/local/system-local-ic-15-Data.db
(245 bytes) for commitlog position ReplayPosition(segmentId=1385570469088, position=52887)
 
Nov 27 16:41:11 xaviring1 cassandra: Compacting [SSTableReader(path='/mnt/cassandra/data/system/local/system-local-ic-12-Data.db'),
SSTableReader(path='/mnt/cassandra/data/system/local/system-local-ic-13-Data.db'), SSTableReader(path='/mnt/cassandra/data/system/local/system-local-ic-14-Data.db'),
SSTableReader(path='/mnt/cassandra/data/system/local/system-local-ic-15-Data.db'), SSTableReader(path='/mnt/cassandra/data/system/local/system-local-ic-11-Data.db')]
 
Nov 27 16:41:11 xaviring1 cassandra: Compacted 5 sstables to [/mnt/cassandra/data/system/local/system-local-ic-16,].
 11,616 bytes to 5,748 (~49% of original) in 529ms = 0.010362MB/s.  5 total rows, 1 unique.
 Row merge counts were {1:0, 2:0, 3:0, 4:0, 5:1, }  
Nov 27 16:41:12 xaviring1 cassandra: Filtering out TLS_RSA_WITH_AES_256_CBC_SHA as it isnt
supported by the socket  
Nov 27 16:41:12 xaviring1 cassandra: Starting Encrypted Messaging Service on SSL port 7000
 
Nov 27 16:41:12 xaviring1 cassandra: Using saved token [-1025783784642367561, -123766659789318196,
-1295312444089505247, -1362940946402062950, -1505674377441592984, -1510151953022964931, -1595956917215185478,
-1618024116320884346, -1661847706648338640, -1718952508753916570, -1723563561873974918, -1771501414901960857,
-1998768013558002607, -2032719025450193591, -2062350291372813732, -216371939563228238, -2214918226708625680,
-2219986229292502225, -2224304248500816743, -2229405184252594928, -2245632632301636047, -2262650862561623702,
-2303010585556132278, -2544761475287241917, -2549894955562210213, -2550092719946622364, -2554549736787432603,
-2634414036433968806, -2672468728734129175, -2719614996320648573, -27...
Nov 27 16:41:12 xaviring1 cassandra: Enqueuing flush of Memtable-local@209168200(84/840 serialized/live
bytes, 4 ops)  
Nov 27 16:41:12 xaviring1 cassandra: Writing Memtable-local@209168200(84/840 serialized/live
bytes, 4 ops)  
Nov 27 16:41:12 xaviring1 cassandra: Completed flushing /mnt/cassandra/data/system/local/system-local-ic-17-Data.db
(120 bytes) for commitlog position ReplayPosition(segmentId=1385570469088, position=53162)
 
Nov 27 16:41:12 xaviring1 cassandra: Enqueuing flush of Memtable-local@60652275(10098/100980
serialized/live bytes, 257 ops)  
Nov 27 16:41:12 xaviring1 cassandra: Writing Memtable-local@60652275(10098/100980 serialized/live
bytes, 257 ops)  
Nov 27 16:41:12 xaviring1 cassandra: Completed flushing /mnt/cassandra/data/system/local/system-local-ic-18-Data.db
(5421 bytes) for commitlog position ReplayPosition(segmentId=1385570469088, position=65169)
 
Nov 27 16:41:12 xaviring1 cassandra: Node /10.114.18.206 state jump to normal  
Nov 27 16:41:12 xaviring1 cassandra: Startup completed! Now serving reads.  
Nov 27 16:41:12 xaviring1 cassandra: mx4j successfuly loaded  
Nov 27 16:41:13 xaviring1 cassandra: Starting listening for CQL clients on /0.0.0.0:9042...
 
Nov 27 16:41:13 xaviring1 cassandra: Binding thrift service to /0.0.0.0:9160  
Nov 27 16:41:13 xaviring1 cassandra: Using TFramedTransport with a max frame size of 15728640
bytes.  
Nov 27 16:41:13 xaviring1 cassandra: Using synchronous/threadpool thrift server on 0.0.0.0
: 9160  
Nov 27 16:41:13 xaviring1 cassandra: Listening for thrift clients...  

Mime
View raw message