Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 42212 invoked from network); 5 Apr 2011 13:41:21 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 5 Apr 2011 13:41:21 -0000 Received: (qmail 72993 invoked by uid 500); 5 Apr 2011 13:41:19 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 72969 invoked by uid 500); 5 Apr 2011 13:41:19 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 72961 invoked by uid 99); 5 Apr 2011 13:41:19 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Apr 2011 13:41:19 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of jbellis@gmail.com designates 209.85.212.44 as permitted sender) Received: from [209.85.212.44] (HELO mail-vw0-f44.google.com) (209.85.212.44) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Apr 2011 13:41:11 +0000 Received: by vws12 with SMTP id 12so339892vws.31 for ; Tue, 05 Apr 2011 06:40:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type:content-transfer-encoding; bh=M85WNx0xidlH+mWmVpsGifU1zM4vz2SbBcLhaF2a66o=; b=HxwG/bUsUXnSccK3MUvE5wbHt3uVbdvBNmqZToAGlyH14hJhSt3+JWgj5/WeuHdhxI jtjXO5c/UanNPXlflsSBHMSumw/u7CDWQoOIRirb/txdGOgykUmVT3b7MzB6Azow82fv KrcHodZjyYP0LFUD26XwaVK2z2WCebMPEBfD8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; b=Aq3QdrS1Vvzn5EeS2+16TQY9H/JBoIAnxDro303rOpfT0gqt9TNXJTHh1NRpdEztCH SeDY1Pmhv9dkw1RLTgsNV5TSuuf7y1x1kq1oSzjRSrqGo6r+2FxS5ywv9tckUwiJgN5n x/tShtj4Faq9+91113jofPwAdqNIeQ3qDgxNo= Received: by 10.52.100.2 with SMTP id eu2mr5221631vdb.20.1302010851136; Tue, 05 Apr 2011 06:40:51 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.169.228 with HTTP; Tue, 5 Apr 2011 06:40:31 -0700 (PDT) In-Reply-To: <120CB7532EA53A4D8CA6B63F94B4ADB351E6D627DB@IE2RD2XVS021.red002.local> References: <120CB7532EA53A4D8CA6B63F94B4ADB351E6D62754@IE2RD2XVS021.red002.local> <120CB7532EA53A4D8CA6B63F94B4ADB351E6D627DB@IE2RD2XVS021.red002.local> From: Jonathan Ellis Date: Tue, 5 Apr 2011 08:40:31 -0500 Message-ID: Subject: Re: too many open files - maybe a fd leak in indexslicequeries To: Roland Gude Cc: "user@cassandra.apache.org" , Juergen Link , Johannes Hoerle Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org sounds like they haven't been munmapped yet. try forcing a GC. On Sat, Apr 2, 2011 at 5:38 AM, Roland Gude wro= te: > > Hi, > > The open file limit is 1024 > Sstable count is somewhere around 20 or so thread count is in the same or= der of magnitude I guess > But lsof shows that deleted sstables still have open file handles. This s= eems to be the issue as this number keeps growing. > Any ideas? > > Roland. > > -----Urspr=FCngliche Nachricht----- > Von: Jonathan Ellis [mailto:jbellis@gmail.com] > Gesendet: Freitag, 1. April 2011 06:07 > An: user@cassandra.apache.org > Cc: Roland Gude; Juergen Link; Johannes Hoerle > Betreff: Re: too many open files - maybe a fd leak in indexslicequeries > > Index queries (ColumnFamilyStore.scan) don't do any low-level i/o > themselves, they go through CFS.getColumnFamily, which is what normal > row fetches also go through. =A0So if there is a leak there it's > unlikely to be specific to indexes. > > What is your open-file limit (remember that sockets count towards > this), thread count, sstable count? > > On Thu, Mar 31, 2011 at 4:15 PM, Roland Gude = wrote: >> I experience something that looks exactly like >> https://issues.apache.org/jira/browse/CASSANDRA-1178 >> >> On cassandra 0.7.3 when using index slice queries (lots of them) >> >> Crashing multiple nodes and rendering the cluster useless. But I have no >> clue where to look if index queries still leak fd >> >> >> >> Does anybody know about it? >> >> Where could I look? >> >> >> >> Greetings, >> >> roland >> >> >> >> -- >> >> YOOCHOOSE GmbH >> >> >> >> Roland Gude >> >> Software Engineer >> >> >> >> Im Mediapark 8, 50670 K=F6ln >> >> >> >> +49 221 4544151 (Tel) >> >> +49 221 4544159 (Fax) >> >> +49 171 7894057 (Mobil) >> >> >> >> >> >> Email:=A0roland.gude@yoochoose.com >> >> WWW: www.yoochoose.com >> >> >> >> YOOCHOOSE GmbH >> >> Gesch=E4ftsf=FChrer: Dr. Uwe Alkemper, Michael Friedmann >> >> Handelsregister: Amtsgericht K=F6ln HRB 65275 >> >> Ust-Ident-Nr: DE 264 773 520 >> >> Sitz der Gesellschaft: K=F6ln >> >> > > > > -- > Jonathan Ellis > Project Chair, Apache Cassandra > co-founder of DataStax, the source for professional Cassandra support > http://www.datastax.com > > > --=20 Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com