From user-return-26678-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Mon Jun 4 21:58:43 2012 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A83D2CECA for ; Mon, 4 Jun 2012 21:58:43 +0000 (UTC) Received: (qmail 38382 invoked by uid 500); 4 Jun 2012 21:58:41 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 38360 invoked by uid 500); 4 Jun 2012 21:58:41 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 38348 invoked by uid 99); 4 Jun 2012 21:58:41 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Jun 2012 21:58:41 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [216.82.241.83] (HELO mail37.messagelabs.com) (216.82.241.83) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 04 Jun 2012 21:58:33 +0000 X-Env-Sender: Bryce.Godfrey@azaleos.com X-Msg-Ref: server-16.tower-37.messagelabs.com!1338847090!16228168!1 X-Originating-IP: [96.31.162.44] X-StarScan-Version: 6.5.10; banners=-,-,- X-VirusChecked: Checked Received: (qmail 15739 invoked from network); 4 Jun 2012 21:58:11 -0000 Received: from unknown (HELO smtp.azaleos.net) (96.31.162.44) by server-16.tower-37.messagelabs.com with AES128-SHA encrypted SMTP; 4 Jun 2012 21:58:11 -0000 Received: from FISH-MBX-02.azaleos.net ([fe80::4918:2597:a7be:1075]) by FISH-HUB-01.azaleos.net ([fe80::b519:b6d:a20d:3073%11]) with mapi id 14.02.0298.004; Mon, 4 Jun 2012 14:58:10 -0700 From: Bryce Godfrey To: "user@cassandra.apache.org" Subject: RE: 1.1 not removing commit log files? Thread-Topic: 1.1 not removing commit log files? Thread-Index: Ac03JfqyldO2HRvSTZKoocBa1/nYjgACBryQAA+WdoAABQZHsAAxTneAAAQ/ONABx3mQUAAYGyqAAB7NZYAAmhUpgAAG5Aog Date: Mon, 4 Jun 2012 21:58:09 +0000 Message-ID: <054A19B0E41FDD43A58D2CC1E8C3FF5B0BDC1546@FISH-MBX-02.azaleos.net> References: <054A19B0E41FDD43A58D2CC1E8C3FF5B0BDA6A8A@FISH-MBX-01.azaleos.net> <0B2BF1E8E35731438C02772C683FB67B11F180AE@AMXPRD0610MB353.eurprd06.prod.outlook.com> <054A19B0E41FDD43A58D2CC1E8C3FF5B0BDA7A83@FISH-MBX-01.azaleos.net> <054A19B0E41FDD43A58D2CC1E8C3FF5B0BDB9702@FISH-MBX-02.azaleos.net> <054A19B0E41FDD43A58D2CC1E8C3FF5B0BDBFA65@FISH-MBX-02.azaleos.net> <4AEF585F-D914-423A-AB6C-5EA27FEA09C4@thelastpickle.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.20.0.100] Content-Type: multipart/alternative; boundary="_000_054A19B0E41FDD43A58D2CC1E8C3FF5B0BDC1546FISHMBX02azaleo_" MIME-Version: 1.0 --_000_054A19B0E41FDD43A58D2CC1E8C3FF5B0BDC1546FISHMBX02azaleo_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I'll try to get some log files for this with DEBUG enabled. Tough on produ= ction though. From: aaron morton [mailto:aaron@thelastpickle.com] Sent: Monday, June 04, 2012 11:15 AM To: user@cassandra.apache.org Subject: Re: 1.1 not removing commit log files? Apply the local hint mutation follows the same code path and regular mutati= ons. When the commit log is being truncated you should see flush activity, logge= d from the ColumnFamilyStore with "Enqueuing flush of " messages. If you set DEBUG logging for the org.apache.cassandra.db.ColumnFamilyStore= it will log if it things the CF is clean and no flush takes place. If you set DEBUG logging on org.apache.cassandra.db.commitlog.CommitLog we = will see if the commit log file could not be deleted because a dirty CF was= not flushed. Cheers A ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 2/06/2012, at 4:43 AM, Rob Coli wrote: On Thu, May 31, 2012 at 7:01 PM, aaron morton > wrote: But that talks about segments not being cleared at startup. Does not explai= n why they were allowed to get past the limit in the first place. Perhaps the commit log size tracking for this limit does not, for some reason, track hints? This seems like the obvious answer given the state which appears to trigger it? This doesn't explain why the files aren't getting deleted after the hints are delivered, of course... =3DRob -- =3DRobert Coli AIM>ALK - rcoli@palominodb.com YAHOO - rcoli.palominob SKYPE - rcoli_palominodb --_000_054A19B0E41FDD43A58D2CC1E8C3FF5B0BDC1546FISHMBX02azaleo_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

I’ll try to get some log files for this with DEBUG enabled.  = Tough on production though.

 <= /p>

From: aaron mo= rton [mailto:aaron@thelastpickle.com]
Sent: Monday, June 04, 2012 11:15 AM
To: user@cassandra.apache.org
Subject: Re: 1.1 not removing commit log files?

 

Apply the local hint mutation follows the same code = path and regular mutations. 

 

When the commit log is being truncated you should se= e flush activity, logged from the ColumnFamilyStore with "Enqueuing fl= ush of " messages. 

 

If you set DEBUG logging for the  org.apache.ca= ssandra.db.ColumnFamilyStore it will log if it things the CF is clean and n= o flush takes place. 

 

If you set DEBUG logging on org.apache.cassandr= a.db.commitlog.CommitLog we will see if the commit log file could not be de= leted because a dirty CF was not flushed. 

 

Cheers

A

 

 

-----------------

Aaron Morton

Freelance Developer<= /o:p>

@aaronmorton

 

On 2/06/2012, at 4:43 AM, Rob Coli wrote:=



On Thu, May 31, 2012 at 7:01 PM, aaron morton <aaron@thelastpickle.com> wr= ote:

But that talks about segments not being cleared at s= tartup. Does not explain

why they were allowed to get past the limit in the f= irst place.


Perhaps the commit log size tracking for this limit does not, for some
reason, track hints? This seems like the obvious answer given the
state which appears to trigger it? This doesn't explain why the files
aren't getting deleted after the hints are delivered, of course...

=3DRob

--
=3DRobert Coli
AIM&GTALK - rcoli@palominodb.co= m
YAHOO - rcoli.palominob
SKYPE - rcoli_palominodb

 

--_000_054A19B0E41FDD43A58D2CC1E8C3FF5B0BDC1546FISHMBX02azaleo_--