Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 79221 invoked from network); 31 May 2010 19:06:46 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 31 May 2010 19:06:46 -0000 Received: (qmail 72076 invoked by uid 500); 31 May 2010 19:06:45 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 72055 invoked by uid 500); 31 May 2010 19:06:45 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 72047 invoked by uid 99); 31 May 2010 19:06:45 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 31 May 2010 19:06:45 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of rantav@gmail.com designates 209.85.211.181 as permitted sender) Received: from [209.85.211.181] (HELO mail-yw0-f181.google.com) (209.85.211.181) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 31 May 2010 19:06:38 +0000 Received: by ywh11 with SMTP id 11so2895310ywh.7 for ; Mon, 31 May 2010 12:06:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:content-type; bh=mDb+Q44JqQggb+WL3s4btTBPGMEBwIM0RpXgzWVgEX8=; b=OmWfg2oXTjzpWl4kA7znAXF0W3pNZUZmJk6q+tCT7s/elsVwhI+MSv2acmHgJRCTAn PsJm5mnxbu5BkNmzfpe9W2BrgBgiSxHPJpZ/VVgAHdV9zq0rLAnoBZLhco5FFuLY1VEO BfVFBxmSbBH2bOJqPJN+v0bMrRnK1qT2GFvDE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=VPmYDrAPcTDIPQQziRyebkz18JAuL1I6gSrzXm6DH4bERbVpraY3BRxNdZp7oe3IxZ gfK5pjbj5M4bFQJZjgda1qPk/SkqQeunIKo8cYeiUgODz29sVg390y4yLR5MRW1s61hF fwUOuSzEJ1iT8QPcisrO9nu/SKsPEUcKoth5M= Received: by 10.231.148.200 with SMTP id q8mr6354942ibv.10.1275332777172; Mon, 31 May 2010 12:06:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.231.172.79 with HTTP; Mon, 31 May 2010 12:05:57 -0700 (PDT) In-Reply-To: References: From: Ran Tavory Date: Mon, 31 May 2010 22:05:57 +0300 Message-ID: Subject: Re: HintedHandoffEnabled To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=0016e6407eba2762ed0487e88d6e X-Virus-Checked: Checked by ClamAV on apache.org --0016e6407eba2762ed0487e88d6e Content-Type: text/plain; charset=UTF-8 I deleted all files in the system dir, including hints before starting the servers. It's not a big deal. but can it be something else? On Mon, May 31, 2010 at 9:53 PM, Jonathan Ellis wrote: > disabling hinted handoff will prevent creation of new hints, but it > will continue to attempt to deliver existing ones unless you manually > delete the hint files in the system/ dir. > > On Mon, May 31, 2010 at 6:04 AM, Ran Tavory wrote: > > In 0.6.2 I disabled hinted handoff, however tpstats and cfstats report > seems > > odd. > > On all servers in the cluster I have: > > false > > tpstats reports 5 completed handoffs. > > $ nodetool -h cass25 -p 9004 tpstats > > Pool Name Active Pending Completed > > FILEUTILS-DELETE-POOL 0 0 2 > > STREAM-STAGE 0 0 0 > > RESPONSE-STAGE 0 0 5903099 > > ROW-READ-STAGE 0 0 669093 > > LB-OPERATIONS 0 0 0 > > MESSAGE-DESERIALIZER-POOL 1 0 6595504 > > GMFD 0 0 35947 > > LB-TARGET 0 0 0 > > CONSISTENCY-MANAGER 0 0 669095 > > ROW-MUTATION-STAGE 0 0 644360 > > MESSAGE-STREAMING-POOL 0 0 0 > > LOAD-BALANCER-STAGE 0 0 0 > > FLUSH-SORTER-POOL 0 0 0 > > MEMTABLE-POST-FLUSHER 0 0 7 > > FLUSH-WRITER-POOL 0 0 7 > > AE-SERVICE-STAGE 0 0 1 > > HINTED-HANDOFF-POOL 0 0 5 > > In data/system/* there are only LocationInfo files, so looks like hinted > > handoff is indeed disabled and cfstats does indicate there are 0 bytes, > > however it also indicates of 32 reads which I didn't expect (cluster has > > been up for a few hours). > > $ nodetool -h cass25 -p 9004 tpstats > > ... > > Column Family: HintsColumnFamily > > SSTable count: 0 > > Space used (live): 0 > > Space used (total): 0 > > Memtable Columns Count: 0 > > Memtable Data Size: 0 > > Memtable Switch Count: 0 > > Read Count: 32 > > Read Latency: 0.062 ms. > > Write Count: 0 > > Write Latency: NaN ms. > > Pending Tasks: 0 > > Key cache capacity: 1 > > Key cache size: 0 > > Key cache hit rate: NaN > > Row cache: disabled > > Compacted row minimum size: 0 > > Compacted row maximum size: 0 > > Compacted row mean size: 0 > > Any idea why is this happening? > > > > -- > Jonathan Ellis > Project Chair, Apache Cassandra > co-founder of Riptano, the source for professional Cassandra support > http://riptano.com > --0016e6407eba2762ed0487e88d6e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: base64 PGRpdiBkaXI9Imx0ciI+SSBkZWxldGVkIGFsbCBmaWxlcyBpbiB0aGUgc3lzdGVtIGRpciwgaW5j bHVkaW5nIGhpbnRzIGJlZm9yZSBzdGFydGluZyB0aGUgc2VydmVycy7CoDxkaXY+SXQmIzM5O3Mg bm90IGEgYmlnIGRlYWwuIGJ1dCBjYW4gaXQgYmUgc29tZXRoaW5nIGVsc2U/PGJyPjxicj48ZGl2 IGNsYXNzPSJnbWFpbF9xdW90ZSI+T24gTW9uLCBNYXkgMzEsIDIwMTAgYXQgOTo1MyBQTSwgSm9u YXRoYW4gRWxsaXMgPHNwYW4gZGlyPSJsdHIiPiZsdDs8YSBocmVmPSJtYWlsdG86amJlbGxpc0Bn bWFpbC5jb20iPmpiZWxsaXNAZ21haWwuY29tPC9hPiZndDs8L3NwYW4+IHdyb3RlOjxicj4KCjxi bG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIgc3R5bGU9Im1hcmdpbjowIDAgMCAuOGV4O2Jv cmRlci1sZWZ0OjFweCAjY2NjIHNvbGlkO3BhZGRpbmctbGVmdDoxZXg7Ij5kaXNhYmxpbmcgaGlu dGVkIGhhbmRvZmYgd2lsbCBwcmV2ZW50IGNyZWF0aW9uIG9mIG5ldyBoaW50cywgYnV0IGl0PGJy Pgp3aWxsIGNvbnRpbnVlIHRvIGF0dGVtcHQgdG8gZGVsaXZlciBleGlzdGluZyBvbmVzIHVubGVz cyB5b3UgbWFudWFsbHk8YnI+CmRlbGV0ZSB0aGUgaGludCBmaWxlcyBpbiB0aGUgc3lzdGVtLyBk aXIuPGJyPgo8ZGl2PjxkaXY+PC9kaXY+PGRpdiBjbGFzcz0iaDUiPjxicj4KT24gTW9uLCBNYXkg MzEsIDIwMTAgYXQgNjowNCBBTSwgUmFuIFRhdm9yeSAmbHQ7PGEgaHJlZj0ibWFpbHRvOnJhbnRh dkBnbWFpbC5jb20iPnJhbnRhdkBnbWFpbC5jb208L2E+Jmd0OyB3cm90ZTo8YnI+CiZndDsgSW4g MC42LjIgSSBkaXNhYmxlZCBoaW50ZWQgaGFuZG9mZiwgaG93ZXZlciB0cHN0YXRzIGFuZCBjZnN0 YXRzIHJlcG9ydCBzZWVtczxicj4KJmd0OyBvZGQuPGJyPgomZ3Q7IE9uIGFsbCBzZXJ2ZXJzIGlu IHRoZSBjbHVzdGVyIEkgaGF2ZTo8YnI+CiZndDsgwqAmbHQ7SGludGVkSGFuZG9mZkVuYWJsZWQm Z3Q7ZmFsc2UmbHQ7L0hpbnRlZEhhbmRvZmZFbmFibGVkJmd0Ozxicj4KJmd0OyB0cHN0YXRzIHJl cG9ydHMgNSBjb21wbGV0ZWQgaGFuZG9mZnMuPGJyPgomZ3Q7ICQgbm9kZXRvb2wgLWggY2FzczI1 IC1wIDkwMDQgdHBzdGF0czxicj4KJmd0OyBQb29sIE5hbWUgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqBBY3RpdmUgwqAgUGVuZGluZyDCoCDCoCDCoENvbXBsZXRlZDxicj4KJmd0OyBGSUxF VVRJTFMtREVMRVRFLVBPT0wgwqAgwqAgwqAgwqAgwqAgwqAgMCDCoCDCoCDCoCDCoCAwIMKgIMKg IMKgIMKgIMKgIMKgIMKgMjxicj4KJmd0OyBTVFJFQU0tU1RBR0UgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAwIMKgIMKgIMKgIMKgIDAgwqAgwqAgwqAgwqAgwqAgwqAgwqAwPGJyPgom Z3Q7IFJFU1BPTlNFLVNUQUdFIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgMCDCoCDCoCDC oCDCoCAwIMKgIMKgIMKgIMKgNTkwMzA5OTxicj4KJmd0OyBST1ctUkVBRC1TVEFHRSDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoDAgwqAgwqAgwqAgwqAgMCDCoCDCoCDCoCDCoCA2NjkwOTM8 YnI+CiZndDsgTEItT1BFUkFUSU9OUyDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCAwIMKg IMKgIMKgIMKgIDAgwqAgwqAgwqAgwqAgwqAgwqAgwqAwPGJyPgomZ3Q7IE1FU1NBR0UtREVTRVJJ QUxJWkVSLVBPT0wgwqAgwqAgwqAgwqAgMSDCoCDCoCDCoCDCoCAwIMKgIMKgIMKgIMKgNjU5NTUw NDxicj4KJmd0OyBHTUZEIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgMCDCoCDCoCDCoCDCoCAwIMKgIMKgIMKgIMKgIMKgMzU5NDc8YnI+CiZndDsgTEItVEFSR0VU IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIDAgwqAgwqAgwqAgwqAgMCDCoCDC oCDCoCDCoCDCoCDCoCDCoDA8YnI+CiZndDsgQ09OU0lTVEVOQ1ktTUFOQUdFUiDCoCDCoCDCoCDC oCDCoCDCoCDCoCAwIMKgIMKgIMKgIMKgIDAgwqAgwqAgwqAgwqAgNjY5MDk1PGJyPgomZ3Q7IFJP Vy1NVVRBVElPTi1TVEFHRSDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoDAgwqAgwqAgwqAgwqAgMCDC oCDCoCDCoCDCoCA2NDQzNjA8YnI+CiZndDsgTUVTU0FHRS1TVFJFQU1JTkctUE9PTCDCoCDCoCDC oCDCoCDCoCDCoDAgwqAgwqAgwqAgwqAgMCDCoCDCoCDCoCDCoCDCoCDCoCDCoDA8YnI+CiZndDsg TE9BRC1CQUxBTkNFUi1TVEFHRSDCoCDCoCDCoCDCoCDCoCDCoCDCoCAwIMKgIMKgIMKgIMKgIDAg wqAgwqAgwqAgwqAgwqAgwqAgwqAwPGJyPgomZ3Q7IEZMVVNILVNPUlRFUi1QT09MIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIDAgwqAgwqAgwqAgwqAgMCDCoCDCoCDCoCDCoCDCoCDCoCDCoDA8YnI+ CiZndDsgTUVNVEFCTEUtUE9TVC1GTFVTSEVSIMKgIMKgIMKgIMKgIMKgIMKgIDAgwqAgwqAgwqAg wqAgMCDCoCDCoCDCoCDCoCDCoCDCoCDCoDc8YnI+CiZndDsgRkxVU0gtV1JJVEVSLVBPT0wgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgMCDCoCDCoCDCoCDCoCAwIMKgIMKgIMKgIMKgIMKgIMKgIMKg Nzxicj4KJmd0OyBBRS1TRVJWSUNFLVNUQUdFIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgMCDC oCDCoCDCoCDCoCAwIMKgIMKgIMKgIMKgIMKgIMKgIMKgMTxicj4KJmd0OyBISU5URUQtSEFORE9G Ri1QT09MIMKgIMKgIMKgIMKgIMKgIMKgIMKgIDAgwqAgwqAgwqAgwqAgMCDCoCDCoCDCoCDCoCDC oCDCoCDCoDU8YnI+CiZndDsgSW4gZGF0YS9zeXN0ZW0vKiB0aGVyZSBhcmUgb25seSBMb2NhdGlv bkluZm8gZmlsZXMsIHNvIGxvb2tzIGxpa2UgaGludGVkPGJyPgomZ3Q7IGhhbmRvZmYgaXMgaW5k ZWVkIGRpc2FibGVkIGFuZCBjZnN0YXRzIGRvZXMgaW5kaWNhdGUgdGhlcmUgYXJlIDAgYnl0ZXMs PGJyPgomZ3Q7IGhvd2V2ZXIgaXQgYWxzbyBpbmRpY2F0ZXMgb2YgMzIgcmVhZHMgd2hpY2ggSSBk aWRuJiMzOTt0IGV4cGVjdCAoY2x1c3RlciBoYXM8YnI+CiZndDsgYmVlbiB1cCBmb3IgYSBmZXcg aG91cnMpLjxicj4KJmd0OyAkIG5vZGV0b29sIC1oIGNhc3MyNSAtcCA5MDA0IHRwc3RhdHM8YnI+ CiZndDsgLi4uPGJyPgomZ3Q7IMKgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBDb2x1bW4gRmFtaWx5 OiBIaW50c0NvbHVtbkZhbWlseTxicj4KJmd0OyDCoMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgU1NU YWJsZSBjb3VudDogMDxicj4KJmd0OyDCoMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgU3BhY2UgdXNl ZCAobGl2ZSk6IDA8YnI+CiZndDsgwqDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoFNwYWNlIHVzZWQg KHRvdGFsKTogMDxicj4KJmd0OyDCoMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgTWVtdGFibGUgQ29s dW1ucyBDb3VudDogMDxicj4KJmd0OyDCoMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgTWVtdGFibGUg RGF0YSBTaXplOiAwPGJyPgomZ3Q7IMKgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBNZW10YWJsZSBT d2l0Y2ggQ291bnQ6IDA8YnI+CiZndDsgwqDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoFJlYWQgQ291 bnQ6IDMyPGJyPgomZ3Q7IMKgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBSZWFkIExhdGVuY3k6IDAu MDYyIG1zLjxicj4KJmd0OyDCoMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgV3JpdGUgQ291bnQ6IDA8 YnI+CiZndDsgwqDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoFdyaXRlIExhdGVuY3k6IE5hTiBtcy48 YnI+CiZndDsgwqDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoFBlbmRpbmcgVGFza3M6IDA8YnI+CiZn dDsgwqDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoEtleSBjYWNoZSBjYXBhY2l0eTogMTxicj4KJmd0 OyDCoMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgS2V5IGNhY2hlIHNpemU6IDA8YnI+CiZndDsgwqDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoEtleSBjYWNoZSBoaXQgcmF0ZTogTmFOPGJyPgomZ3Q7IMKg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBSb3cgY2FjaGU6IGRpc2FibGVkPGJyPgomZ3Q7IMKgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqBDb21wYWN0ZWQgcm93IG1pbmltdW0gc2l6ZTogMDxicj4KJmd0 OyDCoMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgQ29tcGFjdGVkIHJvdyBtYXhpbXVtIHNpemU6IDA8 YnI+CiZndDsgwqDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoENvbXBhY3RlZCByb3cgbWVhbiBzaXpl OiAwPGJyPgomZ3Q7IEFueSBpZGVhIHdoeSBpcyB0aGlzIGhhcHBlbmluZz88YnI+Cjxicj4KPGJy Pgo8YnI+CjwvZGl2PjwvZGl2Pjxmb250IGNvbG9yPSIjODg4ODg4Ij4tLTxicj4KSm9uYXRoYW4g RWxsaXM8YnI+ClByb2plY3QgQ2hhaXIsIEFwYWNoZSBDYXNzYW5kcmE8YnI+CmNvLWZvdW5kZXIg b2YgUmlwdGFubywgdGhlIHNvdXJjZSBmb3IgcHJvZmVzc2lvbmFsIENhc3NhbmRyYSBzdXBwb3J0 PGJyPgo8YSBocmVmPSJodHRwOi8vcmlwdGFuby5jb20iIHRhcmdldD0iX2JsYW5rIj5odHRwOi8v cmlwdGFuby5jb208L2E+PGJyPgo8L2ZvbnQ+PC9ibG9ja3F1b3RlPjwvZGl2Pjxicj48L2Rpdj48 L2Rpdj4K --0016e6407eba2762ed0487e88d6e--