incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thorsten von Eicken <...@rightscale.com>
Subject new node gets no data
Date Thu, 15 Mar 2012 17:45:43 GMT
I added a second node to a single-node ring. RF=1. I can't get the new
node to receive any data. Logs look fine. Here's what nodetool reports:

# nodetool -h localhost ring
Address         DC          Rack        Status State   Load           
Owns    Token
                                                                              
85070591730234615865843651857942052864
10.102.37.168   datacenter1 rack1       Up     Normal  807.81 GB      
50.00%  0
10.80.161.101   datacenter1 rack1       Up     Normal  1.15 MB        
50.00%  85070591730234615865843651857942052864

Just a "little" imbalance. Yes, I use partitioner:
org.apache.cassandra.dht.RandomPartitioner
I tried moving the new node's token up/down by 1 and it triggers the log
messages you'd expect, but no data gets transferred. How do I
troubleshoot this? Below are the log messages I see when restarting the
new node:

 INFO [main] 2012-03-15 17:31:08,616 AbstractCassandraDaemon.java (line
120) JVM vendor/version:
Java HotSpot(TM) 64-Bit Server VM/1.6.0_24
 INFO [main] 2012-03-15 17:31:14,812 CommitLog.java (line 178) Log
replay complete, 8 replayed mutations
 INFO [main] 2012-03-15 17:31:14,825 StorageService.java (line 390)
Cassandra version: 1.0.6
 INFO [main] 2012-03-15 17:31:14,825 StorageService.java (line 391)
Thrift API version: 19.19.0
 INFO [main] 2012-03-15 17:31:14,825 StorageService.java (line 404)
Loading persisted ring state
 INFO [main] 2012-03-15 17:31:14,834 StorageService.java (line 482)
Starting up server gossip
 INFO [main] 2012-03-15 17:31:15,372 MessagingService.java (line 247)
Starting Encrypted Messaging Service on SSL port 7000
 INFO [main] 2012-03-15 17:31:15,376 MessagingService.java (line 268)
Starting Messaging Service on port 7001
 INFO [main] 2012-03-15 17:31:15,401 StorageService.java (line 579)
Using saved token 85070591730234615865843651857942052864
 INFO [main] 2012-03-15 17:31:15,402 ColumnFamilyStore.java (line 692)
Enqueuing flush of Memtable-LocationInfo@645492252(53/66 serialized/live
bytes, 2 ops)
 INFO [FlushWriter:1] 2012-03-15 17:31:15,403 Memtable.java (line 240)
Writing Memtable-LocationInfo@645492252(53/66 serialized/live bytes, 2 ops)
 INFO [FlushWriter:1] 2012-03-15 17:31:15,421 Memtable.java (line 277)
Completed flushing /mnt/ebs/data/system/LocationInfo-hc-32-Data.db (163
bytes)
 INFO [main] 2012-03-15 17:31:15,424 StorageService.java (line 948) Node
/10.80.161.101 state jump to normal
 INFO [main] 2012-03-15 17:31:15,434 StorageService.java (line 589)
Bootstrap/Replace/Move completed! Now serving reads.

# describe keyspace
Keyspace: rslog_production:
  Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
  Durable Writes: true
    Options: [replication_factor:1]
  Column Families:


Mime
View raw message