incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From srmore <comom...@gmail.com>
Subject Re: Multiple data center performance
Date Sat, 08 Jun 2013 13:53:09 GMT
I am seeing the similar behavior, in my case I have 2 nodes in each
datacenter and one node always has high latency (equal to the latency
between the two datacenters). When one of the datacenters is shutdown the
latency drops.

I am curious to know whether anyone else has these issues and if yes how
did to get around it.

Thanks !


On Fri, Jun 7, 2013 at 11:49 PM, Daning Wang <daning@netseer.com> wrote:

> We have deployed multi-center but got performance issue. When the nodes on
> other center are up, the read response time from clients is 4 or 5 times
> higher. when we take those nodes down, the response time becomes
> normal(compare to the time before we changed to multi-center).
>
> We have high volume on the cluster, the consistency level is one for read.
> so my understanding is most of traffic between data center should be read
> repair. but seems that could not create much delay.
>
> What could cause the problem? how to debug this?
>
> Here is the keyspace,
>
> [default@dsat] describe dsat;
> Keyspace: dsat:
>   Replication Strategy:
> org.apache.cassandra.locator.NetworkTopologyStrategy
>   Durable Writes: true
>     Options: [dc2:1, dc1:3]
>   Column Families:
>     ColumnFamily: categorization_cache
>
>
> Ring
>
> Datacenter: dc1
> ===============
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  Address           Load       Tokens  Owns (effective)  Host ID
>                       Rack
> UN  xx.xx.xx..111       59.2 GB    256     37.5%
> 4d6ed8d6-870d-4963-8844-08268607757e  rac1
> DN  xx.xx.xx..121       99.63 GB   256     37.5%
> 9d0d56ce-baf6-4440-a233-ad6f1d564602  rac1
> UN  xx.xx.xx..120       66.32 GB   256     37.5%
> 0fd912fb-3187-462b-8c8a-7d223751b649  rac1
> UN  xx.xx.xx..118       63.61 GB   256     37.5%
> 3c6e6862-ab14-4a8c-9593-49631645349d  rac1
> UN  xx.xx.xx..117       68.16 GB   256     37.5%
> ee6cdf23-d5e4-4998-a2db-f6c0ce41035a  rac1
> UN  xx.xx.xx..116       32.41 GB   256     37.5%
> f783eeef-1c51-4f91-ab7c-a60669816770  rac1
> UN  xx.xx.xx..115       64.24 GB   256     37.5%
> e75105fb-b330-4f40-aa4f-8e6e11838e37  rac1
> UN  xx.xx.xx..112       61.32 GB   256     37.5%
> 2547ee54-88dd-4994-a1ad-d9ba367ed11f  rac1
> Datacenter: dc2
> ===============
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> --  Address           Load       Tokens  Owns (effective)  Host ID
>                       Rack
> DN  xx.xx.xx.199    58.39 GB   256     50.0%
> 6954754a-e9df-4b3c-aca7-146b938515d8  rac1
> DN  xx.xx.xx..61      33.79 GB   256     50.0%
> 91b8d510-966a-4f2d-a666-d7edbe986a1c  rac1
>
>
> Thank you in advance,
>
> Daning
>
>

Mime
View raw message