cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philip Thompson (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-8002) Cassandra SSTableReader hang on startup
Date Tue, 31 Mar 2015 14:37:54 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Philip Thompson updated CASSANDRA-8002:
---------------------------------------
    Description: 
We are experiencing a hang on startup when SSTableReader tries to open certain DataIndex-jb
files, each time its the same SSTable
{code}
#/var/log/system.log
 INFO [SSTableBatchOpen:1] 2014-09-25 07:58:13,660 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25125
(291271971 bytes)
 INFO [SSTableBatchOpen:2] 2014-09-25 07:58:13,660 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25182
(97969101 bytes)
 INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,660 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25217
(8787563 bytes)
 INFO [SSTableBatchOpen:4] 2014-09-25 07:58:13,661 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-23265
(395830480191 bytes)
 INFO [SSTableBatchOpen:5] 2014-09-25 07:58:13,661 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25219
(80412 bytes)
 INFO [SSTableBatchOpen:7] 2014-09-25 07:58:13,663 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25130
(52552441 bytes)
 INFO [SSTableBatchOpen:6] 2014-09-25 07:58:13,663 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25106
(1160245456 bytes)
 INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,663 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25218
(5068542 bytes)
 INFO [SSTableBatchOpen:5] 2014-09-25 07:58:13,737 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25021
(1169149628 bytes)
 INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,745 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25163
(64078835 bytes)
 INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,746 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25187
(53540520 bytes)
 INFO [SSTableBatchOpen:7] 2014-09-25 07:58:13,771 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24537
(18890626389 bytes)
 INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,779 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24936
(1183874336 bytes)
 INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,785 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25158
(99217260 bytes)
 INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,813 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859
(4761451567 bytes){code}
{code}
#/var/log/cassandra.log
 INFO 07:58:13,663 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25106
(1160245456 bytes)
 INFO 07:58:13,663 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25218
(5068542 bytes)
 INFO 07:58:13,737 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25021
(1169149628 bytes)
 INFO 07:58:13,745 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25163
(64078835 bytes)
 INFO 07:58:13,746 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25187
(53540520 bytes)
 INFO 07:58:13,771 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24537
(18890626389 bytes)
 INFO 07:58:13,779 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24936
(1183874336 bytes)
 INFO 07:58:13,785 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25158
(99217260 bytes)
 INFO 07:58:13,813 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859
(4761451567 bytes){code}
{code}
-rw-r--r--. 1 cassandra cassandra 1.4M Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-CompressionInfo.db
-rw-r--r--. 1 cassandra cassandra 4.5G Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Data.db
-rw-r--r--. 1 cassandra cassandra  53M Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Filter.db
-rw-r--r--. 1 cassandra cassandra 3.3G Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Index.db
-rw-r--r--. 1 cassandra cassandra 5.9K Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Statistics.db
-rw-r--r--. 1 cassandra cassandra  26M Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Summary.db
-rw-r--r--. 1 cassandra cassandra   79 Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-TOC.txt{code}

#nodetool netstats{code}
Mode: STARTING
Not sending any streams.
Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
        at com.sun.proxy.$Proxy6.getReadRepairAttempted(Unknown Source)
        at org.apache.cassandra.tools.NodeProbe.getReadRepairAttempted(NodeProbe.java:897)
        at org.apache.cassandra.tools.NodeCmd.printNetworkStats(NodeCmd.java:729)
        at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:1290)
Caused by: javax.management.InstanceNotFoundException: org.apache.cassandra.db:type=StorageProxy
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(Unknown Source)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(Unknown Source)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(Unknown Source)
        at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Source)
        at javax.management.remote.rmi.RMIConnectionImpl.access$300(Unknown Source)
        at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown Source)
        at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown Source)
        at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
        at sun.rmi.transport.Transport$1.run(Unknown Source)
        at sun.rmi.transport.Transport$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Unknown Source)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
        at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(Unknown Source)
        at sun.rmi.transport.StreamRemoteCall.executeCall(Unknown Source)
        at sun.rmi.server.UnicastRef.invoke(Unknown Source)
        at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
        at javax.management.remote.rmi.RMIConnectionImpl_Stub.getAttribute(Unknown Source)
        at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(Unknown
Source)
        at javax.management.MBeanServerInvocationHandler.invoke(Unknown Source)
{code}
# alternatives --config java
There are 2 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
    1           /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java
 +  2           /usr/java/jre1.7.0_60/bin/java

any suggestions?

  was:
We are experiencing a hang on startup when SSTableReader tries to open certain DataIndex-jb
files, each time its the same SSTable

#/var/log/system.log
 INFO [SSTableBatchOpen:1] 2014-09-25 07:58:13,660 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25125
(291271971 bytes)
 INFO [SSTableBatchOpen:2] 2014-09-25 07:58:13,660 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25182
(97969101 bytes)
 INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,660 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25217
(8787563 bytes)
 INFO [SSTableBatchOpen:4] 2014-09-25 07:58:13,661 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-23265
(395830480191 bytes)
 INFO [SSTableBatchOpen:5] 2014-09-25 07:58:13,661 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25219
(80412 bytes)
 INFO [SSTableBatchOpen:7] 2014-09-25 07:58:13,663 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25130
(52552441 bytes)
 INFO [SSTableBatchOpen:6] 2014-09-25 07:58:13,663 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25106
(1160245456 bytes)
 INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,663 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25218
(5068542 bytes)
 INFO [SSTableBatchOpen:5] 2014-09-25 07:58:13,737 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25021
(1169149628 bytes)
 INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,745 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25163
(64078835 bytes)
 INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,746 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25187
(53540520 bytes)
 INFO [SSTableBatchOpen:7] 2014-09-25 07:58:13,771 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24537
(18890626389 bytes)
 INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,779 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24936
(1183874336 bytes)
 INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,785 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25158
(99217260 bytes)
 INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,813 SSTableReader.java (line 223) Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859
(4761451567 bytes)

#/var/log/cassandra.log
 INFO 07:58:13,663 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25106
(1160245456 bytes)
 INFO 07:58:13,663 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25218
(5068542 bytes)
 INFO 07:58:13,737 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25021
(1169149628 bytes)
 INFO 07:58:13,745 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25163
(64078835 bytes)
 INFO 07:58:13,746 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25187
(53540520 bytes)
 INFO 07:58:13,771 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24537
(18890626389 bytes)
 INFO 07:58:13,779 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24936
(1183874336 bytes)
 INFO 07:58:13,785 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25158
(99217260 bytes)
 INFO 07:58:13,813 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859
(4761451567 bytes)

-rw-r--r--. 1 cassandra cassandra 1.4M Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-CompressionInfo.db
-rw-r--r--. 1 cassandra cassandra 4.5G Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Data.db
-rw-r--r--. 1 cassandra cassandra  53M Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Filter.db
-rw-r--r--. 1 cassandra cassandra 3.3G Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Index.db
-rw-r--r--. 1 cassandra cassandra 5.9K Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Statistics.db
-rw-r--r--. 1 cassandra cassandra  26M Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Summary.db
-rw-r--r--. 1 cassandra cassandra   79 Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-TOC.txt

#nodetool netstats
Mode: STARTING
Not sending any streams.
Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
        at com.sun.proxy.$Proxy6.getReadRepairAttempted(Unknown Source)
        at org.apache.cassandra.tools.NodeProbe.getReadRepairAttempted(NodeProbe.java:897)
        at org.apache.cassandra.tools.NodeCmd.printNetworkStats(NodeCmd.java:729)
        at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:1290)
Caused by: javax.management.InstanceNotFoundException: org.apache.cassandra.db:type=StorageProxy
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(Unknown Source)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(Unknown Source)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(Unknown Source)
        at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Source)
        at javax.management.remote.rmi.RMIConnectionImpl.access$300(Unknown Source)
        at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown Source)
        at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown Source)
        at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Unknown Source)
        at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
        at sun.rmi.transport.Transport$1.run(Unknown Source)
        at sun.rmi.transport.Transport$1.run(Unknown Source)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Unknown Source)
        at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
        at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
        at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(Unknown Source)
        at sun.rmi.transport.StreamRemoteCall.executeCall(Unknown Source)
        at sun.rmi.server.UnicastRef.invoke(Unknown Source)
        at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
        at javax.management.remote.rmi.RMIConnectionImpl_Stub.getAttribute(Unknown Source)
        at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(Unknown
Source)
        at javax.management.MBeanServerInvocationHandler.invoke(Unknown Source)

# alternatives --config java
There are 2 programs which provide 'java'.

  Selection    Command
-----------------------------------------------
    1           /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java
 +  2           /usr/java/jre1.7.0_60/bin/java

any suggestions?


> Cassandra SSTableReader hang on startup
> ---------------------------------------
>
>                 Key: CASSANDRA-8002
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8002
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: centos6.5 
> jre-7u60.x86_64
> kernel:2.6.32-431.17.1.el6.x86_64
>            Reporter: Tim Isted
>            Priority: Minor
>             Fix For: 2.0.14
>
>         Attachments: htop.png
>
>
> We are experiencing a hang on startup when SSTableReader tries to open certain DataIndex-jb
files, each time its the same SSTable
> {code}
> #/var/log/system.log
>  INFO [SSTableBatchOpen:1] 2014-09-25 07:58:13,660 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25125 (291271971 bytes)
>  INFO [SSTableBatchOpen:2] 2014-09-25 07:58:13,660 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25182 (97969101 bytes)
>  INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,660 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25217 (8787563 bytes)
>  INFO [SSTableBatchOpen:4] 2014-09-25 07:58:13,661 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-23265 (395830480191 bytes)
>  INFO [SSTableBatchOpen:5] 2014-09-25 07:58:13,661 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25219 (80412 bytes)
>  INFO [SSTableBatchOpen:7] 2014-09-25 07:58:13,663 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25130 (52552441 bytes)
>  INFO [SSTableBatchOpen:6] 2014-09-25 07:58:13,663 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25106 (1160245456 bytes)
>  INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,663 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25218 (5068542 bytes)
>  INFO [SSTableBatchOpen:5] 2014-09-25 07:58:13,737 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25021 (1169149628 bytes)
>  INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,745 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25163 (64078835 bytes)
>  INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,746 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25187 (53540520 bytes)
>  INFO [SSTableBatchOpen:7] 2014-09-25 07:58:13,771 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24537 (18890626389 bytes)
>  INFO [SSTableBatchOpen:3] 2014-09-25 07:58:13,779 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24936 (1183874336 bytes)
>  INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,785 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25158 (99217260 bytes)
>  INFO [SSTableBatchOpen:8] 2014-09-25 07:58:13,813 SSTableReader.java (line 223) Opening
/cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859 (4761451567 bytes){code}
> {code}
> #/var/log/cassandra.log
>  INFO 07:58:13,663 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25106
(1160245456 bytes)
>  INFO 07:58:13,663 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25218
(5068542 bytes)
>  INFO 07:58:13,737 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25021
(1169149628 bytes)
>  INFO 07:58:13,745 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25163
(64078835 bytes)
>  INFO 07:58:13,746 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25187
(53540520 bytes)
>  INFO 07:58:13,771 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24537
(18890626389 bytes)
>  INFO 07:58:13,779 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24936
(1183874336 bytes)
>  INFO 07:58:13,785 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-25158
(99217260 bytes)
>  INFO 07:58:13,813 Opening /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859
(4761451567 bytes){code}
> {code}
> -rw-r--r--. 1 cassandra cassandra 1.4M Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-CompressionInfo.db
> -rw-r--r--. 1 cassandra cassandra 4.5G Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Data.db
> -rw-r--r--. 1 cassandra cassandra  53M Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Filter.db
> -rw-r--r--. 1 cassandra cassandra 3.3G Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Index.db
> -rw-r--r--. 1 cassandra cassandra 5.9K Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Statistics.db
> -rw-r--r--. 1 cassandra cassandra  26M Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-Summary.db
> -rw-r--r--. 1 cassandra cassandra   79 Sep 24 07:23 /cassandra/data/Resources/DataIndex/Resources-DataIndex-jb-24859-TOC.txt{code}
> #nodetool netstats{code}
> Mode: STARTING
> Not sending any streams.
> Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
>         at com.sun.proxy.$Proxy6.getReadRepairAttempted(Unknown Source)
>         at org.apache.cassandra.tools.NodeProbe.getReadRepairAttempted(NodeProbe.java:897)
>         at org.apache.cassandra.tools.NodeCmd.printNetworkStats(NodeCmd.java:729)
>         at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:1290)
> Caused by: javax.management.InstanceNotFoundException: org.apache.cassandra.db:type=StorageProxy
>         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(Unknown Source)
>         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(Unknown
Source)
>         at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(Unknown Source)
>         at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Source)
>         at javax.management.remote.rmi.RMIConnectionImpl.access$300(Unknown Source)
>         at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown
Source)
>         at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown
Source)
>         at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>         at java.lang.reflect.Method.invoke(Unknown Source)
>         at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
>         at sun.rmi.transport.Transport$1.run(Unknown Source)
>         at sun.rmi.transport.Transport$1.run(Unknown Source)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at sun.rmi.transport.Transport.serviceCall(Unknown Source)
>         at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
>         at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
>         at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>         at java.lang.Thread.run(Unknown Source)
>         at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(Unknown Source)
>         at sun.rmi.transport.StreamRemoteCall.executeCall(Unknown Source)
>         at sun.rmi.server.UnicastRef.invoke(Unknown Source)
>         at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
>         at javax.management.remote.rmi.RMIConnectionImpl_Stub.getAttribute(Unknown Source)
>         at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(Unknown
Source)
>         at javax.management.MBeanServerInvocationHandler.invoke(Unknown Source)
> {code}
> # alternatives --config java
> There are 2 programs which provide 'java'.
>   Selection    Command
> -----------------------------------------------
>     1           /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java
>  +  2           /usr/java/jre1.7.0_60/bin/java
> any suggestions?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message