cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain RODRIGUEZ <arodr...@gmail.com>
Subject Re: Default Single DataCenter -> Multi DataCenter
Date Fri, 14 Sep 2018 10:26:09 GMT
Hello,


> Q. Isn't it a problem that at this point, DCAwareRobinPolicy and
> RoundRobinPolicy coexist?


Why not change it on all clients at once? But no, it should not be a
problem as long as you have only 1 DC.

 Q1. Must I add the new seed nodes in five all existing nodes?
>

I generally pick 3 per DC that also become the contact points for the
clients to connect and get information about the whole cluster. Then I just
care about these nodes when removing them (it induces a change in the
client application configuration and on the seeds for all the
cassandra nodes). Change it and restart should be good.

Q2. Don't I need to update the seed node settings of the new cluster
> (mydc2)?
>

All nodes should use the same seed nodes. In this rack use, 2 or 3 nodes
from each rack on all the nodes (no matter the Data Center, each node
references all the seeds nodes).

The procedure looks to be about right, there are probably some details we
are missing while reading but that you will spot while running the test.

C*heers,
-----------------------
Alain Rodriguez - @arodream - alain@thelastpickle.com
France / Spain

The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com

Le mar. 11 sept. 2018 à 11:04, Eunsu Kim <eunsu.bill23@gmail.com> a écrit :

> It’s self respond.
>
> Step3 is wrong.
>
> Even if it was a SimpleSnitch, changing the dc information will not start
> CassandraDaemon with the error log.
>
> ERROR [main] 2018-09-11 18:36:30,272 CassandraDaemon.java:708 - Cannot
> start node if snitch's data center (pg1) differs from previous data center
> (datacenter1). Please fix the snitch configuration, decommission and
> rebootstrap this node or use the flag -Dcassandra.ignore_dc=true.
>
>
> On 11 Sep 2018, at 2:25 PM, Eunsu Kim <eunsu.bill23@gmail.com> wrote:
>
> Hello
>
> Thank you for your responses.
>
> I’ll share my adding datacenter plan. If you see problems, please respond.
>
> The sentence may be a little awkward because I am so poor at English that
> I am being helped by a translator.
>
> I've been most frequently referred to.(https://medium.com/p/465e9bf28d99) Thank
> you for your cleanliness. (Pradeep Chhetri)
>
> I will also upgrade the version as Alain Rodriguez's advice.
>
> ————
>
> Step 1. Stop all existing clusters. (My service is paused.)
>
> Step 2. Install Cassandra 3.11.3 and copy existing conf files.
>
> Step 3. Modify cassandra-rackdc.properties for existing nodes. dc=mydc1
> rack=myrack1
>      Q. I think this modification will not affect the existing data
> because it was SimpleSnitch before, right?
>
> Step 4. In the caseandra.yaml of existing nodes, endpoint_snitch is
> changed to GossippingPropertyFileSnitch.
>
> Step 5. Restart the Cassandra of the existing nodes. (My service is
> resumed.)
>
> Step 6. Change the settings of all existing clients to DCAwareRobinPolicy
> and refer to mydc1. Consistency level is LOCAL_ONE. And rolling restart.
>       Q. Isn't it a problem that at this point, DCAwareRobinPolicy and
> RoundRobinPolicy coexist?
>
> Step 7. Alter my keyspace and system keyspace(system_distributed,
> system_traces) :  SimpleStrategy(RF=2) -> { 'class' :
> 'NetworkTopologyStrategy', ‘mydc1’ : 2 }
>
> Step 8. Install Cassandra in a new cluster, copying existing conf files,
> and setting it to Cassandra-racdc.properties. dc=mydc2 rack=myrack2
>
> Step 9. Adding a new seed node to the cassandra.yaml of the existing
> cluster (mydc1) and restart.
>       Q1. Must I add the new seed nodes in five all existing nodes?
>       Q2. Don't I need to update the seed node settings of the new
> cluster (mydc2)?
>
> Step 10. Alter my keyspace and system keyspace(system_distributed,
> system_traces) :  { 'class' : 'NetworkTopologyStrategy', ‘mydc1’ :
> 1, ‘mydc2’ : 1 }
>
> Step 11. Run 'nodetool rebuild — mydc1’ in turn, in the new node.
>
> ———
>
>
> I'll run the procedure on the development envrionment and share it.
>
> Thank you.
>
>
>
>
> On 10 Sep 2018, at 10:26 PM, Pradeep Chhetri <pradeep@stashaway.com>
> wrote:
>
> Hello Eunsu,
>
> I am going through the same exercise at my job. I was making notes as i
> was testing the steps in my preproduction environment. Although I haven't
> tested end to end but hopefully this might help you:
> https://medium.com/p/465e9bf28d99
>
> Regards,
> Pradeep
>
> On Mon, Sep 10, 2018 at 5:59 PM, Alain RODRIGUEZ <arodrime@gmail.com>
> wrote:
>
>> Adding a data center for the first time is a bit tricky when you
>> haven't been considering it from the start.
>>
>> I operate 5 nodes cluster (3.11.0) in a single data center with
>>> SimpleSnitch, SimpleStrategy and all client policy RoundRobin.
>>>
>>
>> You will need:
>>
>> - To change clients, make them 'DCAware'. This depends on the client, but
>> you should be able to find this in your Cassandra driver (client side).
>> - To change clients, make them use 'LOCAL_<WHATEVER>' consistency
>> ('LOCAL_ONE'/'LOCAL_QUORUM' being the most common).
>> - To change 'SimpleSnitch' for 'EC2Snitch' or
>> 'GossipingPropertyFileSnitch' for example, depending on your
>> context/preference
>> - To change 'SimpleStrategy' for 'NetworkTopologyStrategy' for all the
>> keyspaces, with the desired RF. I take the chance to say that switching to
>> 1 replica only is often a mistake, you can indeed have data loss (which you
>> accept) but also service going down, anytime you restart a node or that a
>> node goes down. If you are ok with RF=1, RDBMS might be a better choice.
>> It's an anti-pattern of some kind to run Cassandra with RF=1. Yet up to
>> you, this is not our topic :). In the same kind of off-topic
>> recommendations, I would not stick with C*3.11.0, but go to C*3.11.3 (if
>> you do not perform slice delete, there is still a bug with this apparently)
>>
>> So this all needs to be done *before* starting adding the new data
>> center. Changing the snitch is tricky, make sure that the new snitch uses
>> the racks and dc names currently in use in your cluster for the current
>> cluster, if not the data could not be accessible after the configuration
>> change.
>>
>> Then the procedure to add a data center is probably described around. I
>> know I did this detailed description in 2014, here it is:
>> https://mail-archives.apache.org/mod_mbox/cassandra-user/201406.mbox/%3CCA+VSrLopop7Th8nX20aOZ3As75g2jrJm3ryX119dekLYNHqFwA@mail.gmail.com%3E,
>> but you might find better/more recent documentation than this one for this
>> relatively common process, like the documentation you linked.
>>
>> If you are not confident or have doubts, you can share more about the
>> context and post your exact plan, as I did years ago in the mail previously
>> linked. People here should be able to confirm the process is ok before you
>> move forward, giving you an extra confidence.
>>
>> C*heers,
>> -----------------------
>> Alain Rodriguez - @arodream - alain@thelastpickle.com
>> France / Spain
>>
>> The Last Pickle - Apache Cassandra Consulting
>> http://www.thelastpickle.com
>>
>> Le lun. 10 sept. 2018 à 11:05, Eunsu Kim <eunsu.bill23@gmail.com> a
>> écrit :
>>
>>> Hello everyone
>>>
>>> I operate 5 nodes cluster (3.11.0) in a single data center with
>>> SimpleSnitch, SimpleStrategy and all client policy RoundRobin.
>>>
>>> At this point, I am going to create clusters of the same size in
>>> different data centers.
>>>
>>> I think these two documents are appropriate, but there is confusion
>>> because they are referenced to each other.
>>>
>>>
>>> https://docs.datastax.com/en/cassandra/3.0/cassandra/operations/opsAddDCToCluster.html
>>>
>>> https://docs.datastax.com/en/cassandra/3.0/cassandra/operations/opsSwitchSnitch.html
>>>
>>> Anyone who can clearly guide the order? Currently RF is 2 and I want to
>>> have only one replica in the NetworkTopologyStrategy.
>>> A little data loss is okay.
>>>
>>> Thank you in advanced..
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>
>
>

Mime
View raw message