Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 64CD39FAC for ; Tue, 22 May 2012 14:22:04 +0000 (UTC) Received: (qmail 73715 invoked by uid 500); 22 May 2012 14:22:01 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 73634 invoked by uid 500); 22 May 2012 14:22:01 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 73624 invoked by uid 99); 22 May 2012 14:22:01 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 May 2012 14:22:01 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of mor.yuki@gmail.com designates 209.85.210.44 as permitted sender) Received: from [209.85.210.44] (HELO mail-pz0-f44.google.com) (209.85.210.44) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 May 2012 14:21:54 +0000 Received: by dacx6 with SMTP id x6so8525244dac.31 for ; Tue, 22 May 2012 07:21:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:message-id:in-reply-to:references:subject:x-mailer :mime-version:content-type; bh=JN+q3SGrWjKmHGic8yyGVg8D2vQmsqFD5+aPb0dI2Ns=; b=XO5SJJcgvGRHlRMKPV0uWBQY8UWlv6y+LoAyjUxruQZVvVl1AMJHt6rblYIB3uJiaZ gt746rAMKfdwRbjhbRCSiSjyEnvXKvDKxY4uPFyeaB9g43yEV0RaFMlfx5FbXxZHkaqY 9K3r7+jHO+9p5xbzC5KmfjmfinPET3zCpnipaN4TsW3JhCQ3mTj9S33flIvOlnaMcrRA 5KEEvCq6jIzyyl3yNMStvZAVlCM9m8byrlHDRC2rauq8Th9/7pXVEoAr42qDpnjeABD5 xUK6xh/g5eK7Vfynxb/DYrbsR9ERt9qw69Aa7XYIvoePdBI4c3JF7C4mt1d0Vb+cZEc3 EBpQ== Received: by 10.68.237.74 with SMTP id va10mr19724453pbc.46.1337696492719; Tue, 22 May 2012 07:21:32 -0700 (PDT) Received: from [192.168.5.113] (ip-64-134-156-131.public.wayport.net. [64.134.156.131]) by mx.google.com with ESMTPS id nw10sm24791077pbb.20.2012.05.22.07.21.31 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 22 May 2012 07:21:31 -0700 (PDT) Date: Tue, 22 May 2012 09:21:29 -0500 From: Yuki Morishita To: user@cassandra.apache.org Message-ID: In-Reply-To: <0B2BF1E8E35731438C02772C683FB67B11F18E42@AMXPRD0610MB353.eurprd06.prod.outlook.com> References: <0B2BF1E8E35731438C02772C683FB67B11F18DAD@AMXPRD0610MB353.eurprd06.prod.outlook.com> <0B2BF1E8E35731438C02772C683FB67B11F18E42@AMXPRD0610MB353.eurprd06.prod.outlook.com> Subject: Re: supercolumns with TTL columns not being compacted correctly X-Mailer: sparrow 1.6 (build 1081.27) MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="4fbba0e9_431bd7b7_2ac8" --4fbba0e9_431bd7b7_2ac8 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Data will not be deleted when those keys appear in other stables outside = of compaction. This is to prevent obsolete data from appearing again. yuki On Tuesday, May 22, 2012 at 7:37 AM, Pieter Callewaert wrote: > =20 > Hi Samal, > =20 > =20 > =20 > =20 > =20 > Thanks for your time looking into this. > =20 > =20 > =20 > =20 > =20 > I force the compaction by using forceUserDefinedCompaction on only that= particular sstable. This gurantees me the new sstable being written only= contains the data from the old sstable. > =20 > =20 > The data in the sstable is more than 31 days old and gc=5Fgrace is 0, b= ut still the data from the sstable is being written to the new one, while= I am 100% sure all the data is invalid. > =20 > =20 > =20 > =20 > =20 > Kind regards, > =20 > =20 > Pieter Callewaert > =20 > =20 > =20 > =20 > =20 > =46rom: samal =5Bmailto:samalgorai=40gmail.com=5D =20 > Sent: dinsdag 22 mei 2012 14:33 > To: user=40cassandra.apache.org (mailto:user=40cassandra.apache.org) > Subject: Re: supercolumns with TTL columns not being compacted correctl= y > =20 > =20 > =20 > =20 > Data will remain till next compaction but won't be available. Compactio= n will delete old sstable create new one. > =20 > On 22-May-2012 5:47 PM, =22Pieter Callewaert=22 wrote: > =20 > =20 > Hi, > =20 > =20 > =20 > =20 > =20 > I=E2=80=99ve had my suspicions some months, but I think I am sure about= it. > =20 > =20 > Data is being written by the SSTableSimpleUnsortedWriter and loaded by = the sstableloader. > =20 > =20 > The data should be alive for 31 days, so I use the following logic: > =20 > =20 > =20 > =20 > =20 > int ttl =3D 2678400; > =20 > =20 > long timestamp =3D System.currentTimeMillis() * 1000; > =20 > =20 > long expirationTimestampMS =3D (long) ((timestamp / 1000) + ((long) ttl= * 1000)); > =20 > =20 > =20 > =20 > =20 > And using this to write it: > =20 > =20 > =20 > =20 > =20 > sstableWriter.newRow(bytes(entry.id (http://entry.id))); > =20 > =20 > sstableWriter.newSuperColumn(bytes(superColumn)); > =20 > =20 > sstableWriter.addExpiringColumn(nameTT, bytes(entry.aggregatedTTMs), ti= mestamp, ttl, expirationTimestampMS); > =20 > =20 > sstableWriter.addExpiringColumn(nameCov, bytes(entry.observationCoverag= e), timestamp, ttl, expirationTimestampMS); > =20 > =20 > sstableWriter.addExpiringColumn(nameSpd, bytes(entry.speed), timestamp,= ttl, expirationTimestampMS); > =20 > =20 > =20 > =20 > =20 > This works perfectly, data can be queried until 31 days are passed, the= n no results are given, as expected. > =20 > =20 > But the data is still on disk until the sstables are being recompacted:= > =20 > =20 > =20 > =20 > =20 > One of our nodes (we got 6 total) has the following sstables: > =20 > =20 > =5Bcassandra=40bemobile-cass3 =7E=5D=24 ls -hal /data/MapData007/HOS-* = =7C grep G > =20 > =20 > -rw-rw-r--. 1 cassandra cassandra 103G May 3 03:19 /data/MapData007/HO= S-hc-125620-Data.db > =20 > =20 > -rw-rw-r--. 1 cassandra cassandra 103G May 12 21:17 /data/MapData007/HO= S-hc-163141-Data.db > =20 > =20 > -rw-rw-r--. 1 cassandra cassandra 25G May 15 06:17 /data/MapData007/HO= S-hc-172106-Data.db > =20 > =20 > -rw-rw-r--. 1 cassandra cassandra 25G May 17 19:50 /data/MapData007/HO= S-hc-181902-Data.db > =20 > =20 > -rw-rw-r--. 1 cassandra cassandra 21G May 21 07:37 /data/MapData007/HO= S-hc-191448-Data.db > =20 > =20 > -rw-rw-r--. 1 cassandra cassandra 6.5G May 21 17:41 /data/MapData007/HO= S-hc-193842-Data.db > =20 > =20 > -rw-rw-r--. 1 cassandra cassandra 5.8G May 22 11:03 /data/MapData007/HO= S-hc-196210-Data.db > =20 > =20 > -rw-rw-r--. 1 cassandra cassandra 1.4G May 22 13:20 /data/MapData007/HO= S-hc-196779-Data.db > =20 > =20 > -rw-rw-r--. 1 cassandra cassandra 401G Apr 16 08:33 /data/MapData007/HO= S-hc-58572-Data.db > =20 > =20 > -rw-rw-r--. 1 cassandra cassandra 169G Apr 16 17:59 /data/MapData007/HO= S-hc-61630-Data.db > =20 > =20 > -rw-rw-r--. 1 cassandra cassandra 173G Apr 17 03:46 /data/MapData007/HO= S-hc-63857-Data.db > =20 > =20 > -rw-rw-r--. 1 cassandra cassandra 105G Apr 23 06:41 /data/MapData007/HO= S-hc-87900-Data.db > =20 > =20 > =20 > =20 > =20 > As you can see, the following files should be invalid: > =20 > =20 > /data/MapData007/HOS-hc-58572-Data.db > =20 > =20 > /data/MapData007/HOS-hc-61630-Data.db > =20 > =20 > /data/MapData007/HOS-hc-63857-Data.db > =20 > =20 > =20 > =20 > =20 > Because they are all written more than an moth ago. gc=5Fgrace is 0 so = this should also not be a problem. > =20 > =20 > =20 > =20 > =20 > As a test, I use forceUserSpecifiedCompaction on the HOS-hc-61630-Data.= db. > =20 > =20 > Expected behavior should be an empty file is being written because all = data in the sstable should be invalid: > =20 > =20 > =20 > =20 > =20 > Compactionstats is giving: > =20 > =20 > compaction type keyspace column family bytes compacted byt= es total progress > =20 > =20 > Compaction MapData007 HOS 115182156= 62 532355279724 2.16% > =20 > =20 > =20 > =20 > =20 > And when I ls the directory I find this: > =20 > =20 > -rw-rw-r--. 1 cassandra cassandra 3.9G May 22 14:12 /data/MapData007/HO= S-tmp-hc-196898-Data.db > =20 > =20 > =20 > =20 > =20 > The sstable is being 1-on-1 copied to a new one. What am I missing here= =3F > =20 > =20 > TTL works perfectly, but is it giving a problem because it is in a supe= r column, and so never to be deleted from disk=3F > =20 > =20 > =20 > =20 > =20 > Kind regards > =20 > =20 > Pieter Callewaert =7C Web & IT engineer > =20 > =20 > Be-Mobile NV (http://www.be-mobile.be/) =7C TouringMobilis (http://www= .touringmobilis.be/) > =20 > =20 > Technologiepark 12b - 9052 Ghent - Belgium > =20 > =20 > Tel + 32 9 330 51 80 =7C =46ax + 32 9 330 51 81 =7C Cell + 32 473 777 = 121 > =20 > =20 > =20 > =20 > =20 > =20 > =20 > =20 > =20 > =20 --4fbba0e9_431bd7b7_2ac8 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
Data will not be deleted when those keys appear in o= ther stables outside of compaction. This is to prevent obsolete data from= appearing again.

yuki

=
=20

On Tuesday, May 22, 20= 12 at 7:37 AM, Pieter Callewaert wrote:

<=21--=5Bif gte mso 9=5D> <=21=5Bendif=5D--><=21--=5Bif gte mso 9=5D> <=21=5Bendif=5D-->

Hi Samal,

 

Thanks for your time looking into this.

 

I force the compaction by using forceUserDefinedCompaction on only that particular = sstable. This gurantees me the new sstable being written only contains th= e data from the old sstable.

The data in the sstable is more than 31 days old and gc=5Fgrace is 0, bu= t still the data from the sstable is being written to the new one, while I am 100% sure all the data is invalid.

 

Kind regards,

Pieter Callewaert

 

=46rom:= samal =5Bmailto:samalgorai=40gmail.com=5D
Sent: dinsdag 22 mei 2012 14:33
To: user=40cass= andra.apache.org
Subject: Re: supercolumns with TTL columns not being compacted cor= rectly

 

Data will remain till next compaction but won't be available. Compacti= on will delete old sstable create new one.

On 22-May-2012 5:47 PM, =22Pieter Callewaert=22 <pieter.callewaert=40= be-mobile.be> wrote:

Hi,

 

I=E2=80=99ve had my suspicion= s some months, but I think I am sure about it.

Data is being written by the = SSTableSimpleUnsortedWriter and loaded by the sstableloader.<= /o:p>

The data should be alive for = 31 days, so I use the following logic:

 

int ttl =3D 2678400;

long timestamp =3D System.currentTimeMillis() * = 1000;

long expirationTimestampMS =3D (long) ((timestam= p / 1000) + ((long) ttl * 1000));

 

And using this to write it:

 

sstableWriter.newRow(bytes(entry.id));

sstableWriter.newSuperColumn(bytes(superColumn))= ;

sstableWriter.addExpiringColumn(nameTT, bytes(en= try.aggregatedTTMs), timestamp, ttl, expirationTimestampMS);<= /o:p>

sstableWriter.addExpiringColumn(nameCov, bytes(e= ntry.observationCoverage), timestamp, ttl, expirationTimestampMS);=

sstableWriter.addExpiringColumn(nameSpd, bytes(e= ntry.speed), timestamp, ttl, expirationTimestampMS);

 

This works perfectly, data ca= n be queried until 31 days are passed, then no results are given, as expe= cted.

But the data is still on disk= until the sstables are being recompacted:

 

One of our nodes (we got 6 to= tal) has the following sstables:

=5Bcassandra=40bemobile-cass3= =7E=5D=24 ls -hal /data/MapData007/HOS-* =7C grep G

-rw-rw-r--. 1 cassandra cassa= ndra 103G May  3 03:19 /data/MapData007/HOS-hc-125620-Data.db=

-rw-rw-r--. 1 cassandra cassa= ndra 103G May 12 21:17 /data/MapData007/HOS-hc-163141-Data.db=

-rw-rw-r--. 1 cassandra cassa= ndra  25G May 15 06:17 /data/MapData007/HOS-hc-172106-Data.db=

-rw-rw-r--. 1 cassandra cassa= ndra  25G May 17 19:50 /data/MapData007/HOS-hc-181902-Data.db=

-rw-rw-r--. 1 cassandra cassa= ndra  21G May 21 07:37 /data/MapData007/HOS-hc-191448-Data.db=

-rw-rw-r--. 1 cassandra cassa= ndra 6.5G May 21 17:41 /data/MapData007/HOS-hc-193842-Data.db=

-rw-rw-r--. 1 cassandra cassa= ndra 5.8G May 22 11:03 /data/MapData007/HOS-hc-196210-Data.db=

-rw-rw-r--. 1 cassandra cassa= ndra 1.4G May 22 13:20 /data/MapData007/HOS-hc-196779-Data.db=

-rw-rw-r--. 1 cassandra cassa= ndra 401G Apr 16 08:33 /data/MapData007/HOS-hc-58572-Data.db<= /o:p>

-rw-rw-r--. 1 cassandra cassa= ndra 169G Apr 16 17:59 /data/MapData007/HOS-hc-61630-Data.db<= /o:p>

-rw-rw-r--. 1 cassandra cassa= ndra 173G Apr 17 03:46 /data/MapData007/HOS-hc-63857-Data.db<= /o:p>

-rw-rw-r--. 1 cassandra cassa= ndra 105G Apr 23 06:41 /data/MapData007/HOS-hc-87900-Data.db<= /o:p>

 

As you can see, the following= files should be invalid:

/data/MapData007/HOS-hc-58572= -Data.db

/data/MapData007/HOS-hc-61630= -Data.db

/data/MapData007/HOS-hc-63857= -Data.db

 

Because they are all written = more than an moth ago. gc=5Fgrace is 0 so this should also not be a probl= em.

 

As a test, I use forceUserSpe= cifiedCompaction on the HOS-hc-61630-Data.db.

Expected behavior should be a= n empty file is being written because all data in the sstable should be i= nvalid:

 

Compactionstats is giving:

compaction type  &n= bsp;     keyspace   column family bytes com= pacted     bytes total  progress

     = ;          Compaction &= nbsp;    MapData007      &nb= sp;      HOS     11518215662=     532355279724     2.16%=

 

And when I ls the directory I= find this:

-rw-rw-r--. 1 cassandra cassa= ndra 3.9G May 22 14:12 /data/MapData007/HOS-tmp-hc-196898-Data.db<= o:p>

 

The sstable is being 1-on-1 c= opied to a new one. What am I missing here=3F

TTL works perfectly, but is i= t giving a problem because it is in a super column, and so never to be de= leted from disk=3F

 

Kind regards

Pieter Callewaert =7C Web & IT engineer

 <= span lang=3D=22EN-US=22 style=3D=22color:gray=22>Be-Mobile NV =7C TouringMobilis=

 T= echnologiepark 12b - 9052 Ghent - Belgium

Tel + 3= 2 9 330 51 80 =7C =46ax + 32 9 330 51 81 =7C  Cell + 32 473 777 121<= /span>

 

 =20 =20 =20 =20
=20

--4fbba0e9_431bd7b7_2ac8--