cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain RODRIGUEZ <arodr...@gmail.com>
Subject Re: Why nodetool cleanup should be run sequentially after node joined a cluster
Date Wed, 11 Apr 2018 11:20:58 GMT
I confirm what Christophe said.

I always ran them in parallel without any problem, really. Historically it
was using only one compactor and impact in my clusters have always been
acceptable.

Nonetheless, newer Cassandra versions allow multiple compactor to work in
parallel during cleanup and this can be really harmful - or really
efficient if resources are available and it is not impacting the read and
write operations. If all the nodes run cleanup in parallel, then limiting
the number of threads used per node is really important.

My colleague Anthony described this option here:
http://thelastpickle.com/blog/2017/08/14/limiting-nodetool-parallel-threads.html
.

C*heers,
-----------------------
Alain Rodriguez - @arodream - alain@thelastpickle.com
France / Spain

The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com


2018-04-11 6:04 GMT+01:00 Christophe Schmitz <christophe@instaclustr.com>:

> Hi Mikhail,
>
>
> Nodetool cleanup can add a fair amount of extra load (mostly IO) on your
> Cassandra nodes. Therefore it is recommended to run it during lower cluster
> usage, and one node at a time, in order to limit the impact on your
> cluster. There are no technical limitations that would prevent you to run
> it at the same time. It's just a precaution measure.
>
> Cheers,
> Christophe
>
>
> On 11 April 2018 at 14:49, Mikhail Tsaplin <tsmisher@gmail.com> wrote:
>
>> Hi,
>> In https://docs.datastax.com/en/cassandra/3.0/cassandra/oper
>> ations/opsAddNodeToCluster.html
>> there is recommendation:
>> 6) After all new nodes are running, run nodetool cleanup
>> <https://docs.datastax.com/en/cassandra/3.0/cassandra/tools/toolsCleanup.html>
>>  on each of the previously existing nodes to remove the keys that no
>> longer belong to those nodes. Wait for cleanup to complete on one node
>> before running nodetool cleanup on the next node.
>>
>> I had added a new node to the cluster, and running nodetool cleanup
>> according to this recommendation - but it takes near 10 days to complete on
>> a single node. Is it safe to start it on all nodes?
>>
>
>
>
> --
>
> *Christophe Schmitz - **VP Consulting*
>
> AU: +61 4 03751980 / FR: +33 7 82022899
>
> <https://www.facebook.com/instaclustr>   <https://twitter.com/instaclustr>
>    <https://www.linkedin.com/company/instaclustr>
>
> Read our latest technical blog posts here
> <https://www.instaclustr.com/blog/>. This email has been sent on behalf
> of Instaclustr Pty. Limited (Australia) and Instaclustr Inc (USA). This
> email and any attachments may contain confidential and legally
> privileged information.  If you are not the intended recipient, do not copy
> or disclose its content, but please reply to this email immediately and
> highlight the error to the sender and then immediately delete the message.
>

Mime
View raw message