cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From maneela a <>
Subject Expanding single node to 2 node cluster
Date Wed, 27 Apr 2011 07:07:05 GMT
I had a 2 node cassandra cluster with replication factor 2 and OrderPreservingPartitioner
but we did not provide InitialToken in the configuration files. One of the node was affected
in the recent AWS EBS outage and had been partitioned from cluster. However, I continued
to allowed all write operations to other survived node because I thought AWS could recovered
EBS issues with in 24 hours so Survived node might take care of propagating 2nd replica from
its hinted column family to the bad node when it recovered from EBS issues. Unfortunately
AWS had taken longer than we expected almost 4 days. So instead of recovering 2nd node by
playing hinted CF from node1, I did the following sequence of events in order to get 2nd
node back to cluster
1) shut down cassandra service on good node2) removed all hinted CF files3) Taken EBS snapshot4)
Launched new EBS volumes from above snapshot and mounted them on 2nd node5) Also copied commitlogs
from node1 to node2in other words, I cloned node1 and mounted on node2, my assumption is cluster
with 2 nodes with replication 2 should likely be mirrored images
6) brought up service on both nodes7) I am not seeing both IP address as part of ring when
I ran nodetool command
root@domU-12-31-39-0F-CA-61:/mnt/logs/cassandra# nodetool -h localhost ringAddress      
Status     Load          Range                                    
 Ring10.193.201.139Up         434.77 GB     RVtMj8gWiKG0baPy                
root@ip-10-196-107-47:/data/cassandra/data/system# nodetool -h localhost ringAddress    
  Status     Load          Range                                  
   Ring10.193.201.139Up         434.77 GB     RVtMj8gWiKG0baPy              

I guess this behavior is happening because both nodes are having same data including Locationinfo
CF and commit logs as well. Can someone direct me what should be done here to get both IPs
as part of ring?
View raw message