incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From srmore <comom...@gmail.com>
Subject Re: Consistency level for multi-datacenter setup
Date Tue, 04 Jun 2013 00:41:51 GMT
After some more investigation it does not appear to be the CL issue. Every
time I am starting up the node in other datacenter with 1sec delay my
throughput starts degrading, even with CL=ONE and CL=LOCAL_QUORUM.

I will put the logs on debug and investigate more and report back the
findings.



On Mon, Jun 3, 2013 at 3:37 PM, Hiller, Dean <Dean.Hiller@nrel.gov> wrote:

> Our badness threshold is 0.1 currently(just checked).  Our website used to
> get slow during a slow node time until we rolled our own patch out.
>
> Dean
>
> From: srmore <comomore@gmail.com<mailto:comomore@gmail.com>>
> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <
> user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
> Date: Monday, June 3, 2013 2:31 PM
> To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <
> user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
> Subject: Re: Consistency level for multi-datacenter setup
>
> We observed that as well, please let us know what you find out it would be
> extremely helpful. There is also this property that you can play with  to
> take care of slow nodes
> dynamic_snitch_badness_threshold.
>
>
> http://www.datastax.com/docs/1.1/configuration/node_configuration#dynamic-snitch-badness-threshold
>
> Thanks !
>
>
> On Mon, Jun 3, 2013 at 3:24 PM, Hiller, Dean <Dean.Hiller@nrel.gov<mailto:
> Dean.Hiller@nrel.gov>> wrote:
> Also, we had to put a fix into cassandra so it removed "slow nodes" from
> the list of nodes to read from.  With that fix our QUOROM(not local quorom)
> started working again and would easily take the other DC nodes out of the
> list of reading from for you as well.  I need to circle back to with my
> teammate to check if he got his fix posted to the dev list or not.
>
> Later,
> Dean
>
> From: srmore <comomore@gmail.com<mailto:comomore@gmail.com><mailto:
> comomore@gmail.com<mailto:comomore@gmail.com>>>
> Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org
> ><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>"
<
> user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:
> user@cassandra.apache.org<mailto:user@cassandra.apache.org>>>
> Date: Monday, June 3, 2013 2:09 PM
> To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:
> user@cassandra.apache.org<mailto:user@cassandra.apache.org>>" <
> user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:
> user@cassandra.apache.org<mailto:user@cassandra.apache.org>>>
> Subject: Consistency level for multi-datacenter setup
>
> I am a bit confused when using the consistency level for multi datacenter
> setup. Following is my setup:
>
> I have 4 nodes the way these are set up are
> Node 1 DC 1 - N1DC1
> Node 2 DC 1 - N2DC1
>
> Node 1 DC 2 - N1DC2
> Node 2 DC 2 - N2DC2
>
> I setup a delay in between two datacenters (DC1 and DC2 around 1 sec one
> way)
>
> I am observing that when I use consistency level 2 for some reason the
> coordinate node is picking up the nodes from other datacenter. My
> understanding was that Cassandra picks up nodes which are close by (from
> local datacenter), determined by Gossip but looks like that's not the case.
>
> I found the following comment on Datastax website :
>
> "If using a consistency level of ONE or LOCAL_QUORUM, only the nodes in
> the same data center as the coordinator node must respond to the client
> request in order for the request to succeed."
>
> Does this mean that for multi datacenter we can only use ONE or
> LOCAL_QUORUM if we want to use the local datacenter to avoid cross
> datacenter latency.
>
> I am using the GossipingPropertyFileSnitch.
>
> Thanks !
>
>
>

Mime
View raw message