cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yuji Ito <y...@imagine-orb.com>
Subject Re: WriteTimeoutException When only One Node is Down
Date Fri, 13 Jan 2017 02:15:19 GMT
Hi Shalom,

I also got WriteTimeoutException in my destructive test like your test.

When did you drop a node?
A coordinator node sends a write request to all replicas.
When one of nodes was down while the request is executed, sometimes
WriteTimeOutException happens.

cf. http://www.datastax.com/dev/blog/cassandra-error-handling-done-right

Thanks,
Yuji



On Thu, Jan 12, 2017 at 4:26 PM, Shalom Sagges <shaloms@liveperson.com>
wrote:

> Hi Everyone,
>
> I'm using C* v3.0.9 for a cluster of 3 DCs with RF 3 in each DC. All
> read/write queries are set to consistency LOCAL_QUORUM.
> The relevant keyspace is built as follows:
>
> *CREATE KEYSPACE mykeyspace WITH replication = {'class':
> 'NetworkTopologyStrategy', 'DC1': '3', 'DC2': '3', 'DC3': '3'}  AND
> durable_writes = true;*
>
> I use* Datastax driver 3.0.1*
>
>
> When I performed a resiliency test for the application, each time I
> dropped one node, the client got the following error:
>
>
> com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra
> timeout during write query at consistency TWO (2 replica were required but
> only 1 acknowledged the write)
> at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(
> WriteTimeoutException.java:73)
> at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(
> WriteTimeoutException.java:26)
> at com.datastax.driver.core.DriverThrowables.propagateCause(
> DriverThrowables.java:37)
> at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(
> DefaultResultSetFuture.java:245)
> at com.datastax.driver.core.AbstractSession.execute(
> AbstractSession.java:63)
> at humanclick.ldap.commImpl.siteData.CassandraSiteDataDaoSpring.
> updateJprunDomains(CassandraSiteDataDaoSpring.java:121)
> at humanclick.ldap.commImpl.siteData.CassandraSiteDataDaoSpring.
> createOrUpdate(CassandraSiteDataDaoSpring.java:97)
> at humanclick.ldapAdapter.dataUpdater.impl.SiteDataToLdapUpdater.update(
> SiteDataToLdapUpdater.java:280)
>
>
> After a few seconds the error no longer recurs. I have no idea why there's
> a timeout since there are additional replicas that satisfy the consistency
> level, and I'm more baffled when the error showed *"Cassandra timeout
> during write query at consistency TWO (2 replica were required but only 1
> acknowledged the write)"*
>
> Any ideas?  I'm quite at a loss here.
>
> Thanks!
>
>
>
> Shalom Sagges
> DBA
> T: +972-74-700-4035 <+972%2074-700-4035>
> <http://www.linkedin.com/company/164748> <http://twitter.com/liveperson>
> <http://www.facebook.com/LivePersonInc> We Create Meaningful Connections
>
>
>
> This message may contain confidential and/or privileged information.
> If you are not the addressee or authorized to receive this on behalf of
> the addressee you must not use, copy, disclose or take action based on this
> message or any information herein.
> If you have received this message in error, please advise the sender
> immediately by reply email and delete this message. Thank you.
>

Mime
View raw message