cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Jirsa"<jji...@apache.org>
Subject Re: Nodetool cleanup doesn't work
Date Thu, 11 May 2017 18:08:17 GMT


On 2017-05-10 22:44 (-0700), Jai Bheemsen Rao Dhanwada <jaibheemsen@gmail.com> wrote:

> Hello,
> 
> I am running into an issue where *nodetool cleanup *fails to cleanup data.
> We are running 2.1.16 version of Cassandra.
> 
> 
> [user@host ~]$ nodetool cleanup
> Aborted cleaning up atleast one column family in keyspace user, check
> server logs for more information.
> Aborted cleaning up atleast one column family in keyspace org, check server
> logs for more information.
> error: nodetool failed, check server logs
> -- StackTrace --
> java.lang.RuntimeException: nodetool failed, check server logs
>         at
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:294)
>         at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:206)
> 
> *Logs:*
> 
> INFO  [RMI TCP Connection(17)-x.x.x.x] 2017-05-05 04:04:07,987
> CompactionManager.java:415 - Cleanup cannot run before a node has joined
> the ring
> INFO  [RMI TCP Connection(17)-x.x.x.x] 2017-05-05 04:04:08,010
> CompactionManager.java:415 - Cleanup cannot run before a node has joined
> the ring
> 
> All the nodes in the cluster are up and running. We tried doing a rolling
> restart of all nodes and no luck.
> 
> After looking at the Cassandra JIRA :
> https://issues.apache.org/jira/browse/CASSANDRA-10991 looks like the issue
> is fixed with 2.2.6 and 3.0 version.
> While we have plans to upgrade to the latest versions(which might take
> longer time), does any know if there is any work around to mitigate the
> issue?
> 

Are you running multiple datacenters, and you just removed a specific datacenter from a keyspace
(and that's why you want to run cleanup)? If that's the case, I fear the fix for 10991 isn't
really going to fix it in the way you hope (we may need a follow-up jira). What you'll almost
certainly need to do is remove the data on disk manually, which is quite unfortunate as it'll
require you to stop+delete-data-for-that-keyspace+start each node in the datacenter for which
you removed replication. 

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org


Mime
View raw message