cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlos Rolo <r...@pythian.com>
Subject Re: Slowness in C* cluster after implementing multiple network interface configuration.
Date Wed, 24 May 2017 08:31:17 GMT
It might be a bug.
Cassandra, AFAIK, scans those files for changes and updates the topology
(So you don't need a restart if you change the files). It might be the case
that the absence of the file, is still noticed by Cassandra even if it is
not really used.

I can do a small test to confirm, if so, it is a question of "expected
behaviour" (as in, always leave the file there) vs Bug (It shouldn't care
for files it doesn't use).

If you can always reproduce, feel free to Open a JIRA.

Thanks for the description.

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
<http://linkedin.com/in/carlosjuzarterolo>*
Mobile: +351 918 918 100
www.pythian.com

On Wed, May 24, 2017 at 8:12 AM, Prakash Chauhan <
prakash.chauhan@ericsson.com> wrote:

> Hi All,
>
>
>
> We have a new observation.
>
>
>
> Earlier for implementing multiple network interfaces, we were deleting
> *cassandra-topologies.properties* in the last step (Steps are mentioned
> in mail trail).
>
> The rationale was that because we are using altogether a new
> endpoint_snitch , we don’t require cassandra-topologies.properties file
> anymore.
>
>
>
> Now we have observed that if we don’t delete cassandra-topologies.properties,
> the slowness is not there in the cluster (Even with multiple restarts)
>
>
>
> Is there some relationship between *GossipingPropertyFileSnitch* and
> *cassandra-topologies.properties* ?
>
>
>
> As per my knowledge,  *cassandra-topologies.properties* file is only used
> as a fallback while doing snitch migration. If that’s the case, why does
> Cassandra becomes slow with time ( and after doing multiple restarts )
> after deleting cassandra-topologies.properties ?
>
>
>
>
>
>
>
>
>
> Regards,
>
> Prakash Chauhan.
>
>
>
> *From:* Cogumelos Maravilha [mailto:cogumelosmaravilha@sapo.pt]
> *Sent:* Wednesday, May 24, 2017 12:15 AM
> *To:* user@cassandra.apache.org
> *Subject:* Re: Slowness in C* cluster after implementing multiple network
> interface configuration.
>
>
>
> Hi,
>
> I never used version 2.0.x but I think port 7000 isn't enough.
>
> Try enable:
>
> 7000 inter-node
>
> 7001 SSL inter-node
>
> 9042 CQL
>
> 9160 Thrift is enable in that version
>
>
>
> And
>
> In Cassandra.yaml, add property “broadcast_address”.  = local ipv4
>
> In Cassandra.yaml, change “listen_address” to private IP. = local ipv4
>
>
>
> As a starting point.
>
>
>
> Cheers.
>
>
>
> On 22-05-2017 12:36, Prakash Chauhan wrote:
>
> Hi All ,
>
>
>
> Need Help !!!
>
>
>
> *Setup Details:*
>
> Cassandra 2.0.14
>
> Geo Red setup
>
> ·         DC1 - 3 nodes
>
> ·         DC2 - 3 nodes
>
>
>
>
>
> We were trying to implement multiple network interfaces with Cassandra
> 2.0.14
>
> After doing all the steps mentioned in DataStax doc
> http://docs.datastax.com/en/archived/cassandra/2.0/
> cassandra/configuration/configMultiNetworks.html, we observed that nodes
> were not able to see each other (checked using nodetool status).
>
>
>
> To resolve this issue, we followed the comment
> <https://issues.apache.org/jira/browse/CASSANDRA-9748?focusedCommentId=14903515&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14903515>
> mentioned in the JIRA : CASSANDRA-9748
> <https://issues.apache.org/jira/browse/CASSANDRA-9748>
>
>
>
> Exact steps that we followed are :
>
> --------------------------------
>
> *1.       *Stop Cassandra
>
> *2.       *Add rule to “iptables” to forward all packets on the public
> interface to the private interface.
>
>
>
> COMMAND: # iptables -t nat -A PREROUTING -p tcp -m tcp -d <destination>
> --dport 7000 -j DNAT --to-destination <to-destination>:7000
>
>
>
> *3.       *In Cassandra.yaml, add property “broadcast_address”.
>
> *4.       *In Cassandra.yaml, change “listen_address” to private IP.
>
> *5.       *Clear the data from directory “peers”.
>
> *6.       *Change Snitch to GossipingPropertyFileSnitch.
>
> *7.       *Append following property to the file “/etc/cassandra/conf/cassandra-env.sh”
> to purge gossip state.
>
> JVM_OPTS="$JVM_OPTS -Dcassandra.load_ring_state=false"
>
>
>
> *8.       *Start Cassandra
>
> *9.       *After node has been started, remove following property from
> the file “/etc/cassandra/conf/cassandra-env.sh” (previously added in step
> 7)
>
> JVM_OPTS="$JVM_OPTS -Dcassandra.load_ring_state=false"
>
> *10.   *Delete file “/etc/cassandra/conf/cassandra-topology.properties”
>
>
>
>
>
> Now We have an observation that after multiple restarts of Cassandra on
> multiple nodes, slowness is observed in the cluster.
>
> The problem gets resolved when we revert the steps mentioned above.
>
>
>
> *Do u think there is any step that can cause the problem ?*
>
> We are suspecting Step 2(iptable rule) but not very sure about it.
>
>
>
>
>
> Regards,
>
> Prakash Chauhan.
>
>
>

-- 


--




Mime
View raw message