incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boris Solovyov <boris.solov...@gmail.com>
Subject Nodetool doesn't shows two nodes
Date Tue, 12 Feb 2013 19:56:02 GMT
I've configured 2-node cluster in EC2, key settings as follows:

cluster_name: 'TS'
num_tokens: 256
seed_provider:
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          - seeds: "ec2-23-21-11-193.compute-1.amazonaws.com,
ec2-107-22-114-19.compute-1.amazonaws.com"
listen_address: 10.145.232.190
broadcast_address: ec2-23-21-11-193.compute-1.amazonaws.com
rpc_address: 0.0.0.0
endpoint_snitch: Ec2MultiRegionSnitch

On other node, it is similar, but of course the listen and broadcast
address are different. Now, when I start Cassandra, I see in the logs

INFO 19:35:32,348 JOINING: waiting for ring information

And then after 30 seconds, it says a bunch of things like this:

JOINING: schema complete, ready to bootstrap
JOINING: getting bootstrap token
Enqueuing flush of Memtable...
JOINING: sleeping 30000 ms for pending range setup
JOINING: Starting to bootstrap...
Bootstrap completed! for the tokens [....]

Finally, after some more memtable flushing,

INFO 19:36:32,710 Node /107.22.114.19 state jump to normal
INFO 19:36:32,722 Startup completed! Now serving reads.

Now, I start the other node, and I see basically the same thing in the logs.

Running nodetool status, I see what looks like two single-node clusters!

[root@ip-10-147-171-160 ~]# nodetool status
Datacenter: us-east
===================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address           Load       Tokens  Owns   Host ID
          Rack
UN  107.22.114.19     21 KB      256     100.0%
 f7a24bd2-8cb9-499d-806c-d9e548f34b8d  1a

[root@ip-10-145-232-190 ~]# nodetool status
Datacenter: us-east
===================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address           Load       Tokens  Owns   Host ID
          Rack
UN  23.21.11.193      21 KB      256     100.0%
 9d70f022-03cf-488a-807d-22e991761483  1a

It looks to me like nodes didn't communicate with each other like I thought
they would, and timed out waiting for gossip to tell them which nodes are
in the ring (I'm new to Cassandra, but this is my guess... certainly
30-second timeouts look suspicious). I checked with telnet, and from each
node I can connect to port 7000 on the other node (both on internal and
public IP). I feel like I have made a beginner's mistake. Anyone has a
suggestion where to look next?

- Boris

Mime
View raw message