Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
Date: Thu, 9 Aug 2012 11:58:40 +1000
Message-ID: 
 <CAGYBkn_CT7W98DYLipn2JRHampHTwZ_oE4Xuw8AtsEcdwY8ukw@mail.gmail.com>
Subject: Syncing nodes + Cassandra Data Availability
From: Ben Kaehne <ben.kaehne@sirca.org.au>
To: user@cassandra.apache.org
Cc: Franc Carter <franc.carter@sirca.org.au>,
 David Nelson <david.nelson@sirca.org.au>
Content-Type: multipart/alternative; boundary=14dae9cdcc7108b54004c6cb9134

--14dae9cdcc7108b54004c6cb9134
Content-Type: text/plain; charset=ISO-8859-1

Good morning,

Our application runs on a 3 node cassandra cluster with RF of 3.

We use quorum operations against this cluster in hopes of garunteeing
consistency.

One scenario in which an issue can occur here is:
Out of our 3 nodes, only 2 are up.
We perform a write to say, a new key.
The down node is started again, at the same time, a different node is
brought offline.
At this point. The data we have written above is on one node, but not the
other online node. Meaning quorum reads will fail.

Surely other people have encountered such issue before.

We disabled hinted handoffs originally as to not have to worry about race
conditions of disk space on servers filling up due to piling up handoffs.
Although perhaps this may somewhat aid the situation (although from what I
read, it does not completely remedy the circumstance).

If so, how are you dealing with it?
>From what I understand a read repair (in which we have set to 1.0) will
only be performed on a successful read occurs, in which will not happen
here.

nodetool repair seems rather slow, is manual and does not suit our
situation where data has to be available apon demand.

Regards,

-- 
-Ben

--14dae9cdcc7108b54004c6cb9134
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Good morning,<br><br>Our application runs on a 3 node cassandra cluster wit=
h RF of 3.<br><br>We use quorum operations against this cluster in hopes of=
 garunteeing consistency.<br><br>One scenario in which an issue can occur h=
ere is:<br>
Out of our 3 nodes, only 2 are up.<br>We perform a write to say, a new key.=
<br>The down node is started again, at the same time, a different node is b=
rought offline.<br>At this point. The data we have written above is on one =
node, but not the other online node. Meaning quorum reads will fail.<br>
<br>Surely other people have encountered such issue before. <br><br>We disa=
bled hinted handoffs originally as to not have to worry about race conditio=
ns of disk space on servers filling up due to piling up handoffs. Although =
perhaps this may somewhat aid the situation (although from what I read, it =
does not completely remedy the circumstance).<br>
<br>If so, how are you dealing with it?<br>From what I understand a read re=
pair (in which we have set to 1.0) will only be performed on a successful r=
ead occurs, in which will not happen here.<br><br>nodetool repair seems rat=
her slow, is manual and does not suit our situation where data has to be av=
ailable apon demand.<br>
<br>Regards,<br clear=3D"all"><br>-- <br>-Ben<br>

--14dae9cdcc7108b54004c6cb9134--