cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan McGuire (JIRA)" <j...@apache.org>
Subject [jira] [Reopened] (CASSANDRA-7658) stress connects to all nodes when it shouldn't
Date Fri, 19 Dec 2014 05:38:14 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-7658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ryan McGuire reopened CASSANDRA-7658:
-------------------------------------

This has regressed, there seems to be two issues here:

1) the nodes it's contacting is not restricted to those specified on the command line.
2) the nodes it is contacting is not contacting the broadcast_address, but rather the listen_address.
This fails in a multi-dc EC2 cluster for instance where the listen_address is a private IP.

Here's a log from a multi-dc cluster setup on EC2. I'm running from DC1, and only specifying
another node in DC1. It shows that it's trying to contact nodes in DC2 that a) are not on
in the stress command line and b) is not using the public broadcast IP

{code}
$ ~/fab/stress/cassandra-2.1/tools/bin/cassandra-stress write n=19000000 -schema replication\(strategy=NetworkTopologyStrategy,factor=2\)
-node ec2-54-183-106-83.us-west-1.compute.amazonaws.com
Unable to create stress keyspace: Error constructing replication strategy class
Warming up WRITE with 50000 iterations...
INFO  04:51:32 Using data-center name 'DC2' for DCAwareRoundRobinPolicy (if this is incorrect,
please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
Connected to cluster: cstar_perf 1EEJMMEZHQ
Datatacenter: DC2; Host: /172.31.44.142; Rack: r1
Datatacenter: DC2; Host: /172.31.36.165; Rack: r1
Datatacenter: DC2; Host: /172.31.46.131; Rack: r1
Datatacenter: DC1; Host: ec2-54-183-106-83.us-west-1.compute.amazonaws.com/172.31.1.124; Rack:
r1
Datatacenter: DC1; Host: /172.31.15.205; Rack: r1
INFO  04:51:32 New Cassandra host /172.31.36.165:9042 added
INFO  04:51:32 New Cassandra host /172.31.46.131:9042 added
ERROR 04:51:37 Error creating pool to /172.31.36.165:9042
com.datastax.driver.core.TransportException: [/172.31.36.165:9042] Cannot connect
        at com.datastax.driver.core.Connection.<init>(Connection.java:104) ~[cassandra-driver-core-2.0.5.jar:na]
        at com.datastax.driver.core.PooledConnection.<init>(PooledConnection.java:28)
~[cassandra-driver-core-2.0.5.jar:na]
        at com.datastax.driver.core.Connection$Factory.open(Connection.java:458) ~[cassandra-driver-core-2.0.5.jar:na]
        at com.datastax.driver.core.HostConnectionPool.<init>(HostConnectionPool.java:85)
~[cassandra-driver-core-2.0.5.jar:na]
        at com.datastax.driver.core.SessionManager.replacePool(SessionManager.java:241) ~[cassandra-driver-core-2.0.5.jar:na]
        at com.datastax.driver.core.SessionManager.access$400(SessionManager.java:42) ~[cassandra-driver-core-2.0.5.jar:na]
        at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:273) [cassandra-driver-core-2.0.5.jar:na]
        at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:265) [cassandra-driver-core-2.0.5.jar:na]
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: org.jboss.netty.channel.ConnectTimeoutException: connection timed out: /172.31.36.165:9042
        at org.jboss.netty.channel.socket.nio.NioClientBoss.processConnectTimeout(NioClientBoss.java:137)
~[netty-3.9.0.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:83)
~[netty-3.9.0.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
~[netty-3.9.0.Final.jar:na]
        at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42) ~[netty-3.9.0.Final.jar:na]
        ... 3 common frames omitted
{code} 

> stress connects to all nodes when it shouldn't
> ----------------------------------------------
>
>                 Key: CASSANDRA-7658
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7658
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Tools
>            Reporter: Brandon Williams
>            Assignee: Benedict
>            Priority: Minor
>             Fix For: 2.1.1
>
>         Attachments: 7658.txt, 7658.v2.txt
>
>
> If you tell stress -node 1,2 in cluster with more nodes, stress appears to do ring discovery
and connect to them all anyway (checked via netstat.)  This led to the confusion on CASSANDRA-7567



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message