cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Yudovin <vla...@winguzone.com>
Subject Re: data not replicated on new node
Date Mon, 21 Nov 2016 07:10:12 GMT
&gt;try SELECT for some undoubtedly old data with consistency ALL

It's worth to turn trace on for this query.



Best regards, Vladimir Yudovin, 

Winguzone - Hosted Cloud Cassandra
Launch your cluster in minutes.





---- On Mon, 21 Nov 2016 02:05:35 -0500Vladimir Yudovin &lt;vladyu@winguzone.com&gt;
wrote ----




&gt;can the node "talk" with the others? (i.e. telnet to the other nodes on port 7000).


According to nodetool status all nodes are joined and UP, so they seems to be able talk one
with other (though there still can be connectivity issues). Are all nodes on internal network
(e.g. 10.x.x.x) in the same locations?



&gt;I suspect that ... the old data stored on the first 2 nodes are not replicated on
the new node.

Can you try SELECT for some undoubtedly old data with consistency ALL (just one raw is enough)?



Also the whole event sequence are not clear: you had two nodes with data, then added third.
When did you change replication factor for existing keyspace? Or it was created with factor
three?



Best regards, Vladimir Yudovin, 

Winguzone - Hosted Cloud Cassandra
Launch your cluster in minutes.





---- On Mon, 21 Nov 2016 01:28:54 -0500Shalom Sagges &lt;shaloms@liveperson.com&gt;
wrote ----




I believe the logs should show you what the issue is. 

Also, can the node "talk" with the others? (i.e. telnet to the other nodes on port 7000).








 
Shalom Sagges
 
DBA
 
T: +972-74-700-4035
 

 
 
 
 We Create Meaningful Connections
 
 

 









On Sun, Nov 20, 2016 at 8:50 PM, Bertrand Brelier &lt;bertrand.brelier@gmail.com&gt;
wrote:









This message may contain confidential and/or privileged information. 

If you are not the addressee or authorized to receive this on behalf of the addressee you
must not use, copy, disclose or take action based on this message or any information herein.


If you have received this message in error, please advise the sender immediately by reply
email and delete this message. Thank you.


Hello Jonathan,



No, the new node is not a seed in my cluster. 



When I ran nodetool bootstrap resume

Node is already bootstrapped.



Cheers,



Bertrand




On Sun, Nov 20, 2016 at 1:43 PM, Jonathan Haddad &lt;jon@jonhaddad.com&gt; wrote:

Did you add the new node as a seed? If you did, it wouldn't bootstrap, and you should run
repair. 

On Sun, Nov 20, 2016 at 10:36 AM Bertrand Brelier &lt;bertrand.brelier@gmail.com&gt;
wrote:

Hello everybody,



I am using a 3-node Cassandra cluster with Cassandra 3.0.10.



I recently added a new node (to make it a 3-node cluster).



I am using a replication factor of 3 , so I expected to have a copy of

the same data on each node :



CREATE KEYSPACE mydata WITH replication = {'class': 'SimpleStrategy',

'replication_factor': '3'}  AND durable_writes = true;



But the new node has  less data that the other 2 :



Datacenter: datacenter1

=======================

Status=Up/Down

|/ State=Normal/Leaving/Joining/Moving

--  Address       Load       Tokens       Owns (effective)  Host

ID                               Rack

UN  XXX.XXX.XXX.XXX  53.28 GB   256          100.0% xxxxxx  rack1

UN  XXX.XXX.XXX.XXX  64.7 GB    256          100.0% xxxxxx  rack1

UN  XXX.XXX.XXX.XXX  1.28 GB    256          100.0% xxxxxx  rack1





On the new node :



/XXXXXX/data-6d674a40efab11e5b67e6d75503d5d02/:

total 1.2G



on one of the old nodes :



/XXXXXX/data-6d674a40efab11e5b67e6d75503d5d02/:

total 52G





I am monitoring the amount of data on each node, and they grow at the

same rate. So I suspect that my new data are replicated on the 3 nodes

but the old data stored on the first 2 nodes are not replicated on the

new node.



I ran nodetool repair (on each node, one at a time), but the new node

still does not have a copy of the old data.



Could you please help me understand why the old data is not replicated

to the new node ? Please let me know if you need further information.



Thank you,



Cheers,



Bertrand


















Mime
View raw message