cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuki Morishita (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-12462) NullPointerException in CompactionInfo.getId(CompactionInfo.java:65)
Date Fri, 30 Sep 2016 00:23:21 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15534513#comment-15534513
] 

Yuki Morishita commented on CASSANDRA-12462:
--------------------------------------------

The only case that {{cfm}} in {{CompactionInfo}} be {{null}} is index summary distribution
(see https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/IndexSummaryRedistribution.java#L309
and https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/compaction/CompactionInfo.java#L47).

Index summary distibution runs across tables, so you cannot pass one table metadata.

[~szhou] your patch only takes care of {{getId()}}. You should patch all {{cfm}} access in
{{CompactionInfo}} class and add null check.

> NullPointerException in CompactionInfo.getId(CompactionInfo.java:65)
> --------------------------------------------------------------------
>
>                 Key: CASSANDRA-12462
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12462
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Compaction
>            Reporter: Jonathan DePrizio
>         Attachments: 0001-Fix-NPE-when-running-nodetool-compactionstats.patch
>
>
> Note: The same trace is cited in the last comment of https://issues.apache.org/jira/browse/CASSANDRA-11961
> I've noticed that some of my nodes in my 2.1 cluster have fallen way behind on compactions,
and have huge numbers (thousands) of uncompacted, tiny SSTables (~30MB or so).
> In diagnosing the issue, I've found that "nodetool compactionstats" returns the exception
below.  Restarting cassandra on the node here causes the pending tasks count to jump to ~2000.
 Compactions run properly for about an hour, until this exception occurs again.  Once it occurs,
I see the pending tasks value rapidly drop towards zero, but without any compactions actually
running (the logs show no compactions finishing).  It would seem that this is causing compactions
to fail on this node, which is leading to it running out of space, etc.
> [redacted]# nodetool compactionstats
> xss =  -ea -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar -XX:+UseThreadPriorities
-XX:ThreadPriorityPolicy=42 -Xms12G -Xmx12G -Xmn1000M -Xss255k
> pending tasks: 5
> error: null
> -- StackTrace --
> java.lang.NullPointerException
> 	at org.apache.cassandra.db.compaction.CompactionInfo.getId(CompactionInfo.java:65)
> 	at org.apache.cassandra.db.compaction.CompactionInfo.asMap(CompactionInfo.java:118)
> 	at org.apache.cassandra.db.compaction.CompactionManager.getCompactions(CompactionManager.java:1405)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> 	at java.lang.reflect.Method.invoke(Unknown Source)
> 	at sun.reflect.misc.Trampoline.invoke(Unknown Source)
> 	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> 	at java.lang.reflect.Method.invoke(Unknown Source)
> 	at sun.reflect.misc.MethodUtil.invoke(Unknown Source)
> 	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
> 	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
> 	at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source)
> 	at com.sun.jmx.mbeanserver.PerInterface.getAttribute(Unknown Source)
> 	at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(Unknown Source)
> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(Unknown Source)
> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(Unknown Source)
> 	at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Source)
> 	at javax.management.remote.rmi.RMIConnectionImpl.access$300(Unknown Source)
> 	at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown Source)
> 	at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown Source)
> 	at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Unknown Source)
> 	at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> 	at java.lang.reflect.Method.invoke(Unknown Source)
> 	at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
> 	at sun.rmi.transport.Transport$1.run(Unknown Source)
> 	at sun.rmi.transport.Transport$1.run(Unknown Source)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at sun.rmi.transport.Transport.serviceCall(Unknown Source)
> 	at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
> 	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
> 	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> 	at java.lang.Thread.run(Unknown Source)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message