incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jakub Glapa <jakub.gl...@gmail.com>
Subject Re: Unsuccessful attempt to add a second node to a ring.
Date Wed, 01 Aug 2012 15:49:05 GMT
I found a similar thread from March :
http://www.mail-archive.com/user@cassandra.apache.org/msg21007.html

For me clearing the data and starting from the beginning didn't help.

It's interesting because on my dev environment I was able to add another
node without any problems.

The only difference is that the second node now is in a different data
center. (but I'm not using any different settings, SimpleSnitch)
7000,9160,7199 ports were open between those 2 nodes.

How else can I check if the communication between those 2 nodes is working?
In the logs I see that:
DEBUG [WRITE-NODE1/node1.ip] 2012-07-31 13:50:39,642
OutboundTcpConnection.java (line 206) attempting to connect to
NODE1/node1.ip

So I assume that the communication is somehow established?


--
regards,
Jakub Glapa


On Wed, Aug 1, 2012 at 11:36 AM, Jakub Glapa <jakub.glapa@gmail.com> wrote:

> yes it's the same
>
>
>
> --
> regards,
> pozdrawiam,
> Jakub Glapa
>
>
> On Wed, Aug 1, 2012 at 11:24 AM, Roshni Rajagopal <
> Roshni.Rajagopal@wal-mart.com> wrote:
>
>> Ok, sorry it may not be required,
>> I was thinking of a configuration I had done on my local laptop, where I
>> had aliased my IP address.
>> In that case the directories and jmx port needed to be different.
>>
>> Cluster name is same right?
>>
>>
>> From: Jakub Glapa <jakub.glapa@gmail.com<mailto:jakub.glapa@gmail.com>>
>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <
>> user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
>> To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <
>> user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
>> Subject: Re: Unsuccessful attempt to add a second node to a ring.
>>
>> Hi Roshni,
>> no they are the same, my changes in cassandra.yaml were only in the
>> listen_address, rpc_address, seeds and initial_token field.
>> The rest is exactly the same as on node1.
>>
>> That's how the file looks on node2:
>>
>>
>>
>> cluster_name: 'Test Cluster'
>> initial_token: 85070591730234615865843651857942052864
>> hinted_handoff_enabled: true
>> hinted_handoff_throttle_delay_in_ms: 1
>> authenticator: org.apache.cassandra.auth.AllowAllAuthenticator
>> authority: org.apache.cassandra.auth.AllowAllAuthority
>> partitioner: org.apache.cassandra.dht.RandomPartitioner
>> data_file_directories:
>>     - /data/servers/cassandra_sbe_edtool/cassandra_data/data
>> commitlog_directory:
>> /data/servers/cassandra_sbe_edtool/cassandra_data/commitlog
>> saved_caches_directory:
>> /data/servers/cassandra_sbe_edtool/cassandra_data/saved_caches
>> commitlog_sync: periodic
>> commitlog_sync_period_in_ms: 10000
>> seed_provider:
>>     - class_name: org.apache.cassandra.locator.SimpleSeedProvider
>>       parameters:
>>           - seeds: "NODE1"
>> flush_largest_memtables_at: 0.75
>> reduce_cache_sizes_at: 0.85
>> reduce_cache_capacity_to: 0.6
>> concurrent_reads: 32
>> concurrent_writes: 32
>> memtable_flush_queue_size: 4
>> sliced_buffer_size_in_kb: 64
>> storage_port: 7000
>> ssl_storage_port: 7001
>> listen_address: NODE2
>> rpc_address: NODE2
>> rpc_port: 9160
>> rpc_keepalive: true
>> rpc_server_type: sync
>> thrift_framed_transport_size_in_mb: 15
>> thrift_max_message_length_in_mb: 16
>> incremental_backups: false
>> snapshot_before_compaction: false
>> column_index_size_in_kb: 64
>> in_memory_compaction_limit_in_mb: 64
>> multithreaded_compaction: false
>> compaction_throughput_mb_per_sec: 16
>> compaction_preheat_key_cache: true
>> rpc_timeout_in_ms: 10000
>> endpoint_snitch: org.apache.cassandra.locator.SimpleSnitch
>> dynamic_snitch_update_interval_in_ms: 100
>> dynamic_snitch_reset_interval_in_ms: 600000
>> dynamic_snitch_badness_threshold: 0.1
>> request_scheduler: org.apache.cassandra.scheduler.NoScheduler
>> index_interval: 128
>> encryption_options:
>>     internode_encryption: none
>>     keystore: conf/.keystore
>>     keystore_password: cassandra
>>     truststore: conf/.truststore
>>     truststore_password: cassandra
>>
>>
>>
>>
>> --
>> regards,
>> pozdrawiam,
>> Jakub Glapa
>>
>>
>> On Wed, Aug 1, 2012 at 10:29 AM, Roshni Rajagopal <
>> Roshni.Rajagopal@wal-mart.com<mailto:Roshni.Rajagopal@wal-mart.com>>
>> wrote:
>> Jakub,
>>
>> Have you set the
>> Data, commitlog, saved cache directories to different ones in each yaml
>> file for each node?
>>
>> Regards,
>> Roshni
>>
>>
>> From: Jakub Glapa <jakub.glapa@gmail.com<mailto:jakub.glapa@gmail.com
>> ><mailto:jakub.glapa@gmail.com<mailto:jakub.glapa@gmail.com>>>
>> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org
>> ><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>"
<
>> user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:
>> user@cassandra.apache.org<mailto:user@cassandra.apache.org>>>
>> To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:
>> user@cassandra.apache.org<mailto:user@cassandra.apache.org>>" <
>> user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:
>> user@cassandra.apache.org<mailto:user@cassandra.apache.org>>>
>> Subject: Unsuccessful attempt to add a second node to a ring.
>>
>> Hi Everybody!
>>
>> I'm trying to add a second node to an already operating one node cluster.
>>
>> Some specs:
>> - cassandra 1.0.7
>> - both nodes have a routable listen_address and rpc_address.
>> - Ports are open: (from node2) telnet node1 7000 is successful
>> - Seeds parameter on node2 points to node 1.
>>
>> [node1] nodetool -h localhost ring
>> Address         DC          Rack        Status State   Load
>>  Owns    Token
>> node1.ip     datacenter1 rack1       Up     Normal  74.33 KB
>>  100.00% 0
>>
>> - initial token on node2 was specified
>>
>> I see something like that in the logs on node2:
>>
>> DEBUG [main] 2012-07-31 13:50:38,640 CollationController.java (line 76)
>> collectTimeOrderedData
>>  INFO [main] 2012-07-31 13:50:38,641 StorageService.java (line 667)
>> JOINING: waiting for ring and schema information
>> DEBUG [WRITE-NODE1/node1.ip] 2012-07-31 13:50:39,642
>> OutboundTcpConnection.java (line 206) attempting to connect to
>> NODE1/node1.ip
>> DEBUG [ScheduledTasks:1] 2012-07-31 13:50:40,639 LoadBroadcaster.java
>> (line 86) Disseminating load info ...
>>  INFO [main] 2012-07-31 13:51:08,641 StorageService.java (line 667)
>> JOINING: schema complete, ready to bootstrap
>> DEBUG [main] 2012-07-31 13:51:08,642 StorageService.java (line 554) ...
>> got ring + schema info
>>  INFO [main] 2012-07-31 13:51:08,642 StorageService.java (line 667)
>> JOINING: getting bootstrap token
>> DEBUG [main] 2012-07-31 13:51:08,644 BootStrapper.java (line 138) token
>> manually specified as 85070591730234615865843651857942052864
>> DEBUG [main] 2012-07-31 13:51:08,645 Table.java (line 387) applying
>> mutation of row 4c
>>
>>
>> but it doesn't join the ring:
>>
>> [node2] nodetool -h localhost ring
>> Address         DC          Rack        Status State   Load
>>  Owns    Token
>> node2.ip   datacenter1 rack1       Up     Normal  13.49 KB        100.00%
>> 85070591730234615865843651857942052864
>>
>>
>>
>> I'm attaching the full log from node2 startup in debug mode.
>>
>>
>>
>> PS.
>> When I didn't specified the initial token on node2 I ended up with
>> exception like that:
>> "Exception encountered during startup: No other nodes seen!  Unable to
>> bootstrap.If you intended to start a single-node cluster, you should make
>> sure your broadcast_address (or listen_address) is listed as a seed.
>> Otherwise, you need to determine why the seed being contacted has no
>> knowledge of the rest of the cluster.  Usually, this can be solved by
>> giving all nodes the same seed list."
>>
>>
>> I'm not sure how to proceed now. I found a couple of posts with problems
>> like that but they weren't very useful.
>>
>> --
>> regards,
>> Jakub Glapa
>>
>> This email and any files transmitted with it are confidential and
>> intended solely for the individual or entity to whom they are addressed. If
>> you have received this email in error destroy it immediately. *** Walmart
>> Confidential ***
>>
>> This email and any files transmitted with it are confidential and
>> intended solely for the individual or entity to whom they are addressed. If
>> you have received this email in error destroy it immediately. *** Walmart
>> Confidential ***
>>
>
>

Mime
View raw message