Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of chirayithaj@gmail.com
 designates 209.85.161.44 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=Zc8SB1jCXjzLY9qVXATkR89SiXPf1lfc6Fk0qVVDgHM7H2ooVeqC6aJ3SuhHOC6B4t
         HbkyikhOL3eivq9VLf0sJM+wDmMMMssJL4VfhA+I99FvI6/TY0+GFtgvCuDV5GZVhsvf
         xnO/JpG7IwSs3MAfpFDXQz40+kW3HlCwAwuoI=
MIME-Version: 1.0
In-Reply-To: <AANLkTi=aUqcY3uQ0FP92bJFS=vAUa0QQVx6W3sbAZV+G@mail.gmail.com>
References: <AANLkTimV5oEw6Vm5OKRXnwudY3csye4xsk87L_FtWWm5@mail.gmail.com>
	<AANLkTi=aUqcY3uQ0FP92bJFS=vAUa0QQVx6W3sbAZV+G@mail.gmail.com>
Date: Mon, 7 Feb 2011 15:08:26 -0600
Message-ID: <AANLkTi=wW+vzrAFBEuDg+JV0X+vQmDjifvOtrZ-AFJ5P@mail.gmail.com>
Subject: Re: Best way to detect/fix bitrot today?
From: Anthony John <chirayithaj@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=00248c11dcc90b9fe5049bb7a213

--00248c11dcc90b9fe5049bb7a213
Content-Type: text/plain; charset=ISO-8859-1

Some RAID storage might do it, potentially more efficiently!!

Rhetorical question - Does Cassandra's architecture of reconciling reads
over multiple copies of the same data provide an even more interesting
answer? I submit - YES!

All bitrot protection mechanisms involve some element of redundant storage -
to verify and reconstruct any rot. Cassandra can do this on JBODs with the
appropriate Replication Factor (say > 3). Granted that the total storage in
terms of number of disks might exceed the other alternatives, but at the
lowest tier, using JBODs, the cost might actually be lesser.

Food for thought, or wild imagination ?

-JA

On Mon, Feb 7, 2011 at 2:09 PM, Peter Schuller
<peter.schuller@infidyne.com>wrote:

> > Our application space is such that there is data that might not be read
> for
> > a long time. The data is mostly immutable. How should I approach
> > detecting/solving the bitrot problem? One approach is read data and let
> read
> > repair do the detection, but given the size of data, that does not look
> very
> > efficient.
>
> Note that read-repair is not really intended to repair arbitrary
> corruptions. Unless I'm mistaken, arbitrary corruption, unless it
> triggers a serialization failure that causes row skipping, it's a
> toss-up which version of the data is retained (or both, if the
> corruption is in the key). Given the same key and column timestamp,
> the tie breaker is the volumn value. So depending on whether
> corruption results in a "lesser" or "greater" value, you might get the
> corrupt or non-corrupt data.
>
> > Has anybody solved/workaround this or has any other suggestions to detect
> > and fix bitrot?
>
> My feel/tentative opinion is that the clean fix is for Cassandra to
> support strong checksumming at the sstable level.
>
> Deploying on e.g. ZFS would help a lot with this, but that's a problem
> for deployment on Linux (which is the recommended platform for
> Cassandra).
>
> --
> / Peter Schuller
>

--00248c11dcc90b9fe5049bb7a213
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Some RAID storage might do it, potentially more efficiently!!<div><br></div=
><div>Rhetorical question - Does Cassandra&#39;s architecture of reconcilin=
g reads over multiple copies of the same data provide an even more interest=
ing answer? I submit - YES!</div>
<div><br></div><div>All bitrot protection mechanisms involve some element o=
f redundant storage - to verify and reconstruct any rot. Cassandra can do t=
his on JBODs with the appropriate Replication Factor (say &gt; 3). Granted =
that the total storage in terms of number of disks might exceed the other a=
lternatives, but at the lowest tier, using JBODs, the cost might actually b=
e lesser.</div>
<div><br></div><div>Food for thought, or wild imagination ?</div><div><br><=
/div><div>-JA</div><div><br><div class=3D"gmail_quote">On Mon, Feb 7, 2011 =
at 2:09 PM, Peter Schuller <span dir=3D"ltr">&lt;<a href=3D"mailto:peter.sc=
huller@infidyne.com">peter.schuller@infidyne.com</a>&gt;</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex;"><div class=3D"im">&gt; Our application spac=
e is such that there is data that might not be read for<br>
&gt; a long time. The data is mostly immutable. How should I approach<br>
&gt; detecting/solving the bitrot problem? One approach is read data and le=
t read<br>
&gt; repair do the detection, but given the size of data, that does not loo=
k very<br>
&gt; efficient.<br>
<br>
</div>Note that read-repair is not really intended to repair arbitrary<br>
corruptions. Unless I&#39;m mistaken, arbitrary corruption, unless it<br>
triggers a serialization failure that causes row skipping, it&#39;s a<br>
toss-up which version of the data is retained (or both, if the<br>
corruption is in the key). Given the same key and column timestamp,<br>
the tie breaker is the volumn value. So depending on whether<br>
corruption results in a &quot;lesser&quot; or &quot;greater&quot; value, yo=
u might get the<br>
corrupt or non-corrupt data.<br>
<div class=3D"im"><br>
&gt; Has anybody solved/workaround this or has any other suggestions to det=
ect<br>
&gt; and fix bitrot?<br>
<br>
</div>My feel/tentative opinion is that the clean fix is for Cassandra to<b=
r>
support strong checksumming at the sstable level.<br>
<br>
Deploying on e.g. ZFS would help a lot with this, but that&#39;s a problem<=
br>
for deployment on Linux (which is the recommended platform for<br>
Cassandra).<br>
<br>
--<br>
<font color=3D"#888888">/ Peter Schuller<br>
</font></blockquote></div><br></div>

--00248c11dcc90b9fe5049bb7a213--