incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiller, Dean" <Dean.Hil...@nrel.gov>
Subject Re: dynamic switch works with NetworkSTrategy NOT SimpleStrategy..solution followsŠ.
Date Mon, 11 Mar 2013 18:21:24 GMT
Ticket filed.

https://issues.apache.org/jira/browse/CASSANDRA-5333

Thanks,
Dean

From: Edward Capriolo <edlinuxguru@gmail.com<mailto:edlinuxguru@gmail.com>>
Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Date: Monday, March 11, 2013 9:16 AM
To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: Re: dynamic switch works with NetworkSTrategy NOT SimpleStrategy..solution followsŠ.

You should file a JIRA if dsnitch only works with LOCAL_QUORUM something is very wrong.

On Mon, Mar 11, 2013 at 9:58 AM, Hiller, Dean <Dean.Hiller@nrel.gov<mailto:Dean.Hiller@nrel.gov>>
wrote:
Well, we finally have dynamic switch working.  It seems to switch nodes to a Remote node and
SimpleSTrategy can not deal with that well.  Also, we had to move to CL=QUOROM_LOCAL instead
of QUOROM.

So for those of you that want cassandra to keep performing well when one node starts to get
really slow, make sure you have

 1.  NetworkTopologyStrategy set
 2.  Have CL=QUOROM_LOCAL

We had to dive into the code to figure all that out.  I hope it helps someone else.

Later,
Dean

From: aaron morton <aaron@thelastpickle.com<mailto:aaron@thelastpickle.com><mailto:aaron@thelastpickle.com<mailto:aaron@thelastpickle.com>>>
Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>"
<user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>>
Date: Monday, March 11, 2013 6:48 AM
To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>"
<user@cassandra.apache.org<mailto:user@cassandra.apache.org><mailto:user@cassandra.apache.org<mailto:user@cassandra.apache.org>>>
Subject: Re: has anyone used dynamic snitch at all

Check that read_repair_chance on the CF's is 0.1, not the old 1.0

Wait at least 10 minutes for the DynamicSnitch to re-calculate.

Use the org.apache.cassandra.db:type=DynamicEndpointSnitch MBean to see what scores it has
given the nodes.

Cheers

-----------------
Aaron Morton
Freelance Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 8/03/2013, at 11:40 AM, Edward Capriolo <edlinuxguru@gmail.com<mailto:edlinuxguru@gmail.com><mailto:edlinuxguru@gmail.com<mailto:edlinuxguru@gmail.com>>>
wrote:

dynamic_snitch=true is the default. So it is usually on "wrapping" other snitches. I have
found several scenarios where it does not work exactly as your would expect.

On Fri, Mar 8, 2013 at 2:26 PM, Hiller, Dean <Dean.Hiller@nrel.gov<mailto:Dean.Hiller@nrel.gov><mailto:Dean.Hiller@nrel.gov<mailto:Dean.Hiller@nrel.gov>>>
wrote:
Our test setup

4 nodes, RF=3, reads at CL=QUOROM and we tried CL=TWO
Tell the network card to slow down every packet on node 2
After fixing astyanax to not go to node 2 anymore, we are still seeing cassandra have issues
as it seems to be involving node 2 somehow.  If we take node 2 down, it all speeds back up.

We are trying to get this working such that a slow node in cassandra does not impact our customers.

We are in 1.2.2 and added the following properties….(our properties show PropertyFileSnitch
though I see the keyspace has org.apache.cassandra.locator.SimpleStrategy set probably because
it was created through a tool instead of CLI…shucks)….anyways, I still expected dynamic
snitch to work….

# controls how often to perform the more expensive part of host score
# calculation
dynamic_snitch: true
dynamic_snitch_update_interval_in_ms: 100
# controls how often to reset all host scores, allowing a bad host to
# possibly recover
dynamic_snitch_reset_interval_in_ms: 600000
# if set greater than zero and read_repair_chance is < 1.0, this will allow
# 'pinning' of replicas to hosts in order to increase cache capacity.
# The badness threshold will control how much worse the pinned host has to be
# before the dynamic snitch will prefer other replicas over it.  This is
# expressed as a double which represents a percentage.  Thus, a value of
# 0.2 means Cassandra would continue to prefer the static snitch values
# until the pinned host was 20% worse than the fastest.
dynamic_snitch_badness_threshold: 0.1

Any help appreciated,
Thanks,
Dean




Mime
View raw message