Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (athena.apache.org: domain of rcoli@eventbrite.com
 designates 209.85.215.44 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <A3D1EC874794404EAFD9CA2622448A3C54B365BF@W12112.ldschurch.org>
References: <A3D1EC874794404EAFD9CA2622448A3C54B365BF@W12112.ldschurch.org>
Date: Thu, 11 Sep 2014 10:18:11 -0700
Message-ID: 
 <CAEDUwd1MyAm+yEEEUyfba8pHG7Ly4YyO4TOvtnLLgbG55pGaCQ@mail.gmail.com>
Subject: Re: Detecting bitrot with incremental repair
From: Robert Coli <rcoli@eventbrite.com>
To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Content-Type: multipart/alternative; boundary=089e0122797c5d42060502cd5ad2

--089e0122797c5d42060502cd5ad2
Content-Type: text/plain; charset=UTF-8

On Thu, Sep 11, 2014 at 9:44 AM, John Sumsion <SumsionJG@familysearch.org>
wrote:

> jbellis talked about incremental repair, which is great, but as I
> understood, repair was also somewhat responsible for detecting and
> repairing bitrot on long-lived sstables.
>

SSTable checksums, and the checksums on individual compressed (and only
compressed) partitions provide some of this functionality, at very least
giving some visibility into bitrot style corruption.


> If repair doesn't do it, what will?


Read repair will help, but only repair is capable of providing the
guarantee you need. Probably Cassandra needs partition checksums on
uncompressed partitions, and then to mark a sstable un-repaired when it
detects a corrupt read.

=Rob

--089e0122797c5d42060502cd5ad2
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On T=
hu, Sep 11, 2014 at 9:44 AM, John Sumsion <span dir=3D"ltr">&lt;<a href=3D"=
mailto:SumsionJG@familysearch.org" target=3D"_blank">SumsionJG@familysearch=
.org</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"ma=
rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">jbellis talked=
 about incremental repair, which is great, but as I understood, repair was =
also somewhat responsible for detecting and repairing bitrot on long-lived =
sstables.<br></blockquote><div><br></div><div>SSTable checksums, and the ch=
ecksums on individual compressed (and only compressed) partitions provide s=
ome of this functionality, at very least giving some visibility into bitrot=
 style corruption.</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" =
style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
If repair doesn&#39;t do it, what will?</blockquote><div><br></div><div>Rea=
d repair will help, but only repair is capable of providing the guarantee y=
ou need. Probably Cassandra needs partition checksums on uncompressed parti=
tions, and then to mark a sstable un-repaired when it detects a corrupt rea=
d.</div><div><br></div><div>=3DRob</div><div>=C2=A0</div><div>=C2=A0</div><=
/div></div></div>

--089e0122797c5d42060502cd5ad2--