From user-return-7189-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Thu Jul 01 15:36:14 2010 Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 58023 invoked from network); 1 Jul 2010 15:36:14 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 1 Jul 2010 15:36:14 -0000 Received: (qmail 28878 invoked by uid 500); 1 Jul 2010 15:36:12 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 28834 invoked by uid 500); 1 Jul 2010 15:36:12 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 28821 invoked by uid 99); 1 Jul 2010 15:36:11 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Jul 2010 15:36:11 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of stevelihn@gmail.com designates 209.85.212.44 as permitted sender) Received: from [209.85.212.44] (HELO mail-vw0-f44.google.com) (209.85.212.44) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Jul 2010 15:36:04 +0000 Received: by vws14 with SMTP id 14so887310vws.31 for ; Thu, 01 Jul 2010 08:35:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=k89dRiP+HFTbS3OKAcT3EzMRKCZH654tDyAW6XP0UWY=; b=j6cTBt5XJtFKVlErPopIK5DzTS+VOWCgUs9s7wnFnsvDNLxDQeH1DnIZ6hmCznuA/k mSo7xvCK/Cg6FV0B/IJZmdm1M7K1fhhcM6k7DANk9tPsGa1shil6VCQ3lu8w4zOAvNy0 k7KJbmIt7i2qaSLHE1YQQy3BsyoZOriy9zNaw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=b+sjWMSHG/x2qK45KWoTrrDv7TYvmawhMWHUJH7Y9cc/2Z0lIqdMZLipPbvaJq0zS5 dK1w16XF+H4paHyx7/0sa7IA2gS7pbnKvmzC//5HzjS90MkTHC647x6UzIVfuLbp5MQr B2nBTFDjDsyNWHpA88v+0ATRNYVSkLsH/b66I= MIME-Version: 1.0 Received: by 10.220.126.152 with SMTP id c24mr881331vcs.12.1277998543612; Thu, 01 Jul 2010 08:35:43 -0700 (PDT) Received: by 10.220.202.74 with HTTP; Thu, 1 Jul 2010 08:35:43 -0700 (PDT) In-Reply-To: References: Date: Thu, 1 Jul 2010 11:35:43 -0400 Message-ID: Subject: Re: Compaction-pool pause -- was: Cluster-wide pause From: Steve Lihn To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=001636c93289372a4c048a5539f2 X-Virus-Checked: Checked by ClamAV on apache.org --001636c93289372a4c048a5539f2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable This problem was solved by forming a 3-node large-instance cluster. The pause went away. I thought I would try a single node configuration to test the intensive inserts, as you would expect to just work (though may not performing well). It turns out somehow Cassandra likes to have a minimum amount of clustering. If my experiment is correct, maybe this should be mentioned on the wiki; or else the single node configuration should be improved to at least work smoothly albeit slower. On Wed, Jun 30, 2010 at 3:21 PM, Steve Lihn wrote: > Jon, > Looks like this compaction process took 360 seconds. What is it and how d= o > I avoid it? > > INFO [COMPACTION-POOL:1] 2010-06-30 14:56:08,667 CompactionManager.java > (line 246) Compacting > [org.apache.cassandra.io.SSTableReader(path=3D'/mnt/itops/cdata/TFO/Curre= ntHolding-129-Data.db'),org.apache.cassandra.io.SSTableReader(path=3D'/mnt/= itops/cdata/TFO/CurrentHolding-150-Data.db'),org.apache.cassandra.io.SSTabl= eReader(path=3D'/mnt/itops/cdata/TFO/CurrentHolding-172-Data.db'),org.apach= e.cassandra.io.SSTableReader(path=3D'/mnt/itops/cdata/TFO/CurrentHolding-19= 4-Data.db')] > INFO [COMPACTION-POOL:1] 2010-06-30 15:02:09,537 CompactionManager.java > (line 326) Compacted to /mnt/itops/cdata/TFO/CurrentHolding-197-Data.db. > 1880884459/1880884459 bytes for 6306383 keys. Time: 360870ms. > > On Wed, Jun 30, 2010 at 3:12 PM, Steve Lihn wrote: > >> Jon, >> I am experimenting writing 8 million rows into Cassandra and also >> experienced some random timeouts, even with 10-second timeout parameter. >> How do I avoid such timeout at all cost? (I.e. At this time, my priority >> is to finish the end-to-end test. Don't want the program to fail at all.= ) >> This is a one-node server with ms1G and mx2GB. The rest in >> cassandra.in.sh are default I think. >> >> Steve >> >> >> --001636c93289372a4c048a5539f2 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable This problem was solved by forming a 3-node large-instance cluster. The pau= se went away. I thought I would try a single node configuration to test the= intensive inserts, as you would expect to just work (though may not perfor= ming well). It turns out somehow Cassandra likes to have a minimum amount o= f clustering. If my experiment is correct, maybe this should be mentioned o= n the wiki; or else the single node configuration should be improved to at = least work smoothly albeit slower.

On Wed, Jun 30, 2010 at 3:21 PM, Steve Lihn = <stevelihn@gmai= l.com> wrote:
Jon,
Looks like this compaction process took 360 seconds. What is it and= how do I avoid it?

INFO [COMPACTION-POOL:1] 2010-06-30 14:56:08,667= CompactionManager.java (line 246) Compacting [org.apache.cassandra.io.SSTa= bleReader(path=3D'/mnt/itops/cdata/TFO/CurrentHolding-129-Data.db')= ,org.apache.cassandra.io.SSTableReader(path=3D'/mnt/itops/cdata/TFO/Cur= rentHolding-150-Data.db'),org.apache.cassandra.io.SSTableReader(path=3D= '/mnt/itops/cdata/TFO/CurrentHolding-172-Data.db'),org.apache.cassa= ndra.io.SSTableReader(path=3D'/mnt/itops/cdata/TFO/CurrentHolding-194-D= ata.db')]
=C2=A0INFO [COMPACTION-POOL:1] 2010-06-30 15:02:09,537 CompactionManager.ja= va (line 326) Compacted to /mnt/itops/cdata/TFO/CurrentHolding-197-Data.db.= =C2=A0 1880884459/1880884459 bytes for 6306383 keys.=C2=A0 Time: 360870ms.<= br>
On Wed, Jun 30, 2010 at 3:12 PM, Steve Lihn <stevelihn@gmail.com&g= t; wrote:
Jon,
I am experimenting writing 8 million rows into Cassandra and also e= xperienced some random timeouts, even with 10-second timeout parameter.
= How do I avoid such timeout at all cost? (I.e. At this time, my priority is= to finish the end-to-end test. Don't want the program to fail at all.)=
This is a one-node server with ms1G and mx2GB. The rest in cassandra.in.sh are default I thin= k.

Steve



--001636c93289372a4c048a5539f2--