Return-Path: 
 <user-return-15771-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org>
Delivered-To: apmail-cassandra-user-archive@www.apache.org
Received: (qmail 5545 invoked from network); 13 Apr 2011 13:59:54 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3)
  by minotaur.apache.org with SMTP; 13 Apr 2011 13:59:54 -0000
Received: (qmail 65816 invoked by uid 500); 13 Apr 2011 13:59:52 -0000
Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org
Received: (qmail 65792 invoked by uid 500); 13 Apr 2011 13:59:52 -0000
Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@cassandra.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@cassandra.apache.org>
List-Post: <mailto:user@cassandra.apache.org>
List-Id: <user.cassandra.apache.org>
Reply-To: user@cassandra.apache.org
Delivered-To: mailing list user@cassandra.apache.org
Received: (qmail 65784 invoked by uid 99); 13 Apr 2011 13:59:52 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 Apr 2011 13:59:52 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of tmarthinussen@gmail.com
 designates 74.125.82.44 as permitted sender)
Received: from [74.125.82.44] (HELO mail-ww0-f44.google.com) (74.125.82.44)
    by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 13 Apr 2011 13:59:44 +0000
Received: by wwa36 with SMTP id 36so633754wwa.25
        for <user@cassandra.apache.org>; Wed, 13 Apr 2011 06:59:24 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:mime-version:date:message-id:subject:from:to
         :content-type;
        bh=cbucGsNKP15pTsgJHRLELhSPqXuwH7XxKQ4QDwjgRpI=;
        b=VRh/i3c6cuM/7QgDbNTZM5QK3rt/E0E+Ovsk/SOI0M55HFGvLQUpvSb9D4WnMezc74
         Rdx5qoByd6SKFIM7CYYhZcjri9oeghmuY4lEq22cV4Qlcl7xP7rzGIv/K9MoD0z8fEew
         ejpKgD9DE/qE7wH1AOxv2G/zQlwskB75QZmp4=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=S7Qfwge+Qk2LvIAAa4NYbmZuP8yd1iBdAqnYeX9ffKcRhbltyuB6MRki0JUpxR3Wpi
         H9WY3zZUKYYlEFKThEtL/z8KdMExH31SwHf2FBgu2fbD6n1P2Hy5sJSzme3OcL5No9eo
         G6GIjH9bjMTMN/IkiuuKMGwSVaHEp8F85L/1s=
MIME-Version: 1.0
Received: by 10.216.141.225 with SMTP id g75mr3277275wej.10.1302703164232;
 Wed, 13 Apr 2011 06:59:24 -0700 (PDT)
Received: by 10.216.81.9 with HTTP; Wed, 13 Apr 2011 06:59:24 -0700 (PDT)
Date: Wed, 13 Apr 2011 22:59:24 +0900
Message-ID: <BANLkTikwh+1O_yzh-kcUP33akKTz8esTow@mail.gmail.com>
Subject: value of hinted handoff column not really empty...?
From: Terje Marthinussen <tmarthinussen@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=0016e6dab06b5a15d104a0cd3707
X-Virus-Checked: Checked by ClamAV on apache.org

--0016e6dab06b5a15d104a0cd3707
Content-Type: text/plain; charset=ISO-8859-1

Hi,

we do see occasional row corruptions now and then and especially in hinted
handoffs.

This may be related to fairly long rows (millions of columns)

I was dumping one corrupted hint .db file and I noticed that they do in fact
have values.

The doc say
Subcolumn values are always empty; instead, we store the row data "normally"

The code does
            add(path, ByteBufferUtil.EMPTY_BYTE_BUFFER,
System.currentTimeMillis(), cf.metadata().getGcGraceSeconds());

and if you run sstable2json you will see that columns have values like
 "4d8eb49d",

I guess an EMPTY_BYTE_BUFFER is not entirely an empty value. Not such a big
deal, but it may be that we are wasting 4 bytes per hint here?

Just a curiosity I thought I would mention before I forget it.

Terje

--0016e6dab06b5a15d104a0cd3707
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi,=A0<div><br></div><div>we do see occasional row corruptions now and then=
 and especially in hinted handoffs.</div><div><br></div><div>This may be re=
lated to fairly long rows (millions of columns)</div><div><br></div><div>I =
was dumping one corrupted hint .db file and I noticed that they do in fact =
have values.</div>
<div><br></div><div>The doc say</div><div>Subcolumn values are always empty=
; instead, we store the row data &quot;normally&quot;</div><div><br></div><=
div>The code does</div><div><div>=A0=A0 =A0 =A0 =A0 =A0 =A0add(path, ByteBu=
fferUtil.EMPTY_BYTE_BUFFER, System.currentTimeMillis(), cf.metadata().getGc=
GraceSeconds());</div>
</div><div><br></div><div>and if you run sstable2json you will see that col=
umns have values like</div><div><div>=A0&quot;4d8eb49d&quot;,</div></div><d=
iv><br></div><div>I guess an EMPTY_BYTE_BUFFER is not entirely an empty val=
ue. Not such a big deal, but it may be that we are wasting 4 bytes per hint=
 here?=A0</div>
<div><br></div><div>Just a curiosity I thought I would mention before I for=
get it.</div><div><br></div><div>Terje</div>

--0016e6dab06b5a15d104a0cd3707--