From user-return-34030-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Fri May 10 17:57:01 2013 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BD1F0F904 for ; Fri, 10 May 2013 17:57:01 +0000 (UTC) Received: (qmail 41640 invoked by uid 500); 10 May 2013 17:56:59 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 41613 invoked by uid 500); 10 May 2013 17:56:59 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 41602 invoked by uid 99); 10 May 2013 17:56:59 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 May 2013 17:56:59 +0000 X-ASF-Spam-Status: No, hits=4.0 required=5.0 tests=HTML_MESSAGE,MISSING_SUBJECT,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [98.138.229.118] (HELO nm36-vm6.bullet.mail.ne1.yahoo.com) (98.138.229.118) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 May 2013 17:56:51 +0000 Received: from [98.138.90.56] by nm36.bullet.mail.ne1.yahoo.com with NNFMP; 10 May 2013 17:56:30 -0000 Received: from [98.138.87.12] by tm9.bullet.mail.ne1.yahoo.com with NNFMP; 10 May 2013 17:56:30 -0000 Received: from [127.0.0.1] by omp1012.mail.ne1.yahoo.com with NNFMP; 10 May 2013 17:56:30 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 874927.16782.bm@omp1012.mail.ne1.yahoo.com Received: (qmail 42871 invoked by uid 60001); 10 May 2013 17:56:30 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1368208590; bh=ehj+WgdnV4qMnMcvRd809f1/5UOBVmXHRu7bImaV2Do=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Reply-To:To:MIME-Version:Content-Type; b=VFqfylWvTe2O18J7Ou0kjSYQCjv0sMvBWQtiXJHyyqQFzhtd7987Hx+moORoQzohdb7EY8zbBMSoZncTLRBRnzmY7RTYkWKd8hPGFLmo4ZKr7LbFEjB3sHcspJjCNyXcBI0ar1aXrc7kz+909VjERs0TKkpSQ6bL8MJcI7Gb4TY= DomainKey-Signature:a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Reply-To:To:MIME-Version:Content-Type; b=6NPNn5wX9AJ+h+deFBJ5+49vqRGsYdercKPh/ZOKUil+o9cpuVR8WWCDP+EMi3gaf9nmz16K95v+MdVYra+KS5ZkItW+TrI56AfeAl5m1iP1t6N8txs3v28d2DksTkKKrWt3T59Sb519k0UT9+pauU/rHFrpYThFVoefFEdFF2s=; X-YMail-OSG: HyD33bAVM1lryepIv.19TjucaIzgXeEwL3Lc7mPLFgJq.mi kAXaCy_jLXCLKwO7D34aGnLRMAYQJWeB.CX8ioZSFTrWcZGVrXdvEyl8ab_o k9RikxuqsrkIpiersIpTFTj6LNpOruO22Zsw6rwB4si7AzwF9N2baXC89Z5J AJG6z39kEm_BJDpwIqADZfqEd8Fp5zCCZssyG1Bgbvr52A.7PLfm_bbdSqu4 5js60GfnG5gxxrJkOrazteB0xcCM0e9Aidz68POIw1J7UpvWEDCXOXhLldaa X7qekVsoPJOeACICbU884NQBW235DNE7uq9CIi3fmGzudL2OMwL0h6Cp4XM2 W2ELx2sQtSlhShd9jmWEZE1Z7cE0UBxZXZ9Zr6DmVbo8dzrc_.T.CvtmILOr F9.rZfT5LHU7nKtpoMeN0OEUCcGNFIfN1TTnjRkJ_cc0OLkYVEFjzbWn9zip ynQIIIWnT5AIl_jkweuKIiH._yVa1ysUIW0CanI4g.Efd0kRudtDH8.XoH1J KhAMdg2QV Received: from [209.131.62.116] by web31812.mail.mud.yahoo.com via HTTP; Fri, 10 May 2013 10:56:30 PDT X-Rocket-MIMEInfo: 002.001,TXkgY2x1c3RlciBvZiAxMSBub2RlcyBydW5uaW5nIENhc2FuZHJhIDEuMS01IGlzwqAgcGF1c2luZyB0b28gbG9uZyBmb3IgUGFyTmV3IEdDLCB3aGljaCBpbmNyZWFzZXMgb3VyIHJlc3BvbnNlIGxhdGVuY3ksIElzIGl0IGEgZ29vZCBpZGVhIHRvIGhhdmUgYSBhIHNtYWxsZXIgSEVBUF9ORVdTSVpFIHNvIHRoYXQgd2UgY2FuIGNvbGxlY3QgbW9yZSBvZnRlbiwgYnV0IG5vdCBwYXVzZSB0aGF0IGxvbmc_CgoKSU5GTyBbU2NoZWR1bGVkVGFza3M6MV0gMjAxMy0wNS0xMCAwMTowMDoxNywyNDUgR0NJbnNwZWMBMAEBAQE- X-Mailer: YahooMailWebService/0.8.141.536 Message-ID: <1368208590.23005.YahooMailNeo@web31812.mail.mud.yahoo.com> Date: Fri, 10 May 2013 10:56:30 -0700 (PDT) From: Bao Le Reply-To: Bao Le To: "user@cassandra.apache.org" MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="1458549034-1769865613-1368208590=:23005" X-Virus-Checked: Checked by ClamAV on apache.org --1458549034-1769865613-1368208590=:23005 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable My cluster of 11 nodes running Casandra 1.1-5 is=A0 pausing too long for Pa= rNew GC, which increases our response latency, Is it a good idea to have a = a smaller HEAP_NEWSIZE so that we can collect more often, but not pause tha= t long?=0A=0A=0AINFO [ScheduledTasks:1] 2013-05-10 01:00:17,245 GCInspector= .java (line 122) GC for ParNew: 252 ms for 1 collections, 3238218616 used; = max is 8422162432=0A=A0INFO [ScheduledTasks:1] 2013-05-10 02:15:20,650 GCIn= spector.java (line 122) GC for ParNew: 445 ms for 1 collections, 4810760088= used; max is 8422162432=0A=A0INFO [ScheduledTasks:1] 2013-05-10 02:30:19,9= 32 GCInspector.java (line 122) GC for ParNew: 419 ms for 2 collections, 521= 0373288 used; max is 8422162432=0A=A0INFO [ScheduledTasks:1] 2013-05-10 02:= 40:23,201 GCInspector.java (line 122) GC for ParNew: 333 ms for 1 collectio= ns, 2172614912 used; max is 8422162432=0A=A0INFO [ScheduledTasks:1] 2013-05= -10 03:45:19,975 GCInspector.java (line 122) GC for ParNew: 201 ms for 1 co= llections, 4134399864 used; max is 8422162432=0A=A0INFO [ScheduledTasks:1] = 2013-05-10 03:55:20,345 GCInspector.java (line 122) GC for ParNew: 685 ms f= or 1 collections, 4696326432 used; max is 8422162432=0A=A0INFO [ScheduledTa= sks:1] 2013-05-10 04:40:21,957 GCInspector.java (line 122) GC for ParNew: 3= 79 ms for 1 collections, 4051166216 used; max is 8422162432=0A=A0INFO [Sche= duledTasks:1] 2013-05-10 04:50:23,057 GCInspector.java (line 122) GC for Pa= rNew: 334 ms for 1 collections, 4695497128 used; max is 8422162432=0A=A0INF= O [ScheduledTasks:1] 2013-05-10 05:05:20,304 GCInspector.java (line 122) GC= for ParNew: 222 ms for 1 collections, 5527026728 used; max is 8422162432= =0A=A0INFO [ScheduledTasks:1] 2013-05-10 05:35:21,848 GCInspector.java (lin= e 122) GC for ParNew: 279 ms for 1 collections, 3138206504 used; max is 842= 2162432=0A=A0INFO [ScheduledTasks:1] 2013-05-10 05:45:19,939 GCInspector.ja= va (line 122) GC for ParNew: 353 ms for 1 collections, 3445606832 used; max= is 8422162432=0A=A0INFO [ScheduledTasks:1] 2013-05-10 05:55:21,326 GCInspe= ctor.java (line 122) GC for ParNew: 344 ms for 1 collections, 4331945664 us= ed; max is 8422162432=0A=A0INFO [ScheduledTasks:1] 2013-05-10 06:05:20,424 = GCInspector.java (line 122) GC for ParNew: 214 ms for 1 collections, 478780= 6520 used; max is 8422162432=0A=A0INFO [ScheduledTasks:1] 2013-05-10 07:00:= 21,402 GCInspector.java (line 122) GC for ParNew: 256 ms for 1 collections,= 5119566040 used; max is 8422162432=0A=A0INFO [ScheduledTasks:1] 2013-05-10= 07:15:20,747 GCInspector.java (line 122) GC for ParNew: 512 ms for 2 colle= ctions, 2068901896 used; max is 8422162432=0A=A0INFO [ScheduledTasks:1] 201= 3-05-10 07:30:19,081 GCInspector.java (line 122) GC for ParNew: 267 ms for = 1 collections, 2614774320 used; max is 8422162432=0A=A0INFO [ScheduledTasks= :1] 2013-05-10 08:10:22,440 GCInspector.java (line 122) GC for ParNew: 305 = ms for 1 collections, 4042611368 used; max is 8422162432=0A=A0INFO [Schedul= edTasks:1] 2013-05-10 08:15:20,482 GCInspector.java (line 122) GC for ParNe= w: 371 ms for 1 collections, 4365244824 used; max is 8422162432=0A=A0INFO [= ScheduledTasks:1] 2013-05-10 08:25:20,047 GCInspector.java (line 122) GC fo= r ParNew: 251 ms for 1 collections, 4900957800 used; max is 8422162432=0A= =0AThanks=0ABao=0A --1458549034-1769865613-1368208590=:23005 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable
My cluster of 11= nodes running Casandra 1.1-5 is  pausing too long for ParNew GC, whic= h increases our response latency, Is it a good idea to have a a smaller HEA= P_NEWSIZE so that we can collect more often, but not pause that long?

INFO [ScheduledTasks:1] 2013-05-10 01:00:17,245 GCI= nspector.java (line 122) GC for ParNew: 252 ms for 1 collections, 323821861= 6 used; max is 8422162432
 INFO [ScheduledTasks:1] 2013-05-10 02:15= :20,650 GCInspector.java (line 122) GC for ParNew: 445 ms for 1 collections= , 4810760088 used; max is 8422162432
 INFO [ScheduledTasks:1] 2013-= 05-10 02:30:19,932 GCInspector.java (line 122) GC for ParNew: 419 ms for 2 collections, 5210373288 used; max is 8422162432
 INFO [Scheduled= Tasks:1] 2013-05-10 02:40:23,201 GCInspector.java (line 122) GC for ParNew:= 333 ms for 1 collections, 2172614912 used; max is 8422162432
 INFO= [ScheduledTasks:1] 2013-05-10 03:45:19,975 GCInspector.java (line 122) GC = for ParNew: 201 ms for 1 collections, 4134399864 used; max is 8422162432 INFO [ScheduledTasks:1] 2013-05-10 03:55:20,345 GCInspector.java (li= ne 122) GC for ParNew: 685 ms for 1 collections, 4696326432 used; max is 84= 22162432
 INFO [ScheduledTasks:1] 2013-05-10 04:40:21,957 GCInspect= or.java (line 122) GC for ParNew: 379 ms for 1 collections, 4051166216 used= ; max is 8422162432
 INFO [ScheduledTasks:1] 2013-05-10 04:50:23,05= 7 GCInspector.java (line 122) GC for ParNew: 334 ms for 1 collections, 4695= 497128 used; max is 8422162432
 INFO [ScheduledTasks:1] 2013-05-10 = 05:05:20,304 GCInspector.java (line 122) GC for ParNew: 222 ms for 1 collections, 5527026728 used; max is 8422162432
 INFO [ScheduledTa= sks:1] 2013-05-10 05:35:21,848 GCInspector.java (line 122) GC for ParNew: 2= 79 ms for 1 collections, 3138206504 used; max is 8422162432
 INFO [= ScheduledTasks:1] 2013-05-10 05:45:19,939 GCInspector.java (line 122) GC fo= r ParNew: 353 ms for 1 collections, 3445606832 used; max is 8422162432
&= nbsp;INFO [ScheduledTasks:1] 2013-05-10 05:55:21,326 GCInspector.java (line= 122) GC for ParNew: 344 ms for 1 collections, 4331945664 used; max is 8422= 162432
 INFO [ScheduledTasks:1] 2013-05-10 06:05:20,424 GCInspector= .java (line 122) GC for ParNew: 214 ms for 1 collections, 4787806520 used; = max is 8422162432
 INFO [ScheduledTasks:1] 2013-05-10 07:00:21,402 = GCInspector.java (line 122) GC for ParNew: 256 ms for 1 collections, 511956= 6040 used; max is 8422162432
 INFO [ScheduledTasks:1] 2013-05-10 07= :15:20,747 GCInspector.java (line 122) GC for ParNew: 512 ms for 2 collections, 2068901896 used; max is 8422162432
 INFO [ScheduledTa= sks:1] 2013-05-10 07:30:19,081 GCInspector.java (line 122) GC for ParNew: 2= 67 ms for 1 collections, 2614774320 used; max is 8422162432
 INFO [= ScheduledTasks:1] 2013-05-10 08:10:22,440 GCInspector.java (line 122) GC fo= r ParNew: 305 ms for 1 collections, 4042611368 used; max is 8422162432
&= nbsp;INFO [ScheduledTasks:1] 2013-05-10 08:15:20,482 GCInspector.java (line= 122) GC for ParNew: 371 ms for 1 collections, 4365244824 used; max is 8422= 162432
 INFO [ScheduledTasks:1] 2013-05-10 08:25:20,047 GCInspector= .java (line 122) GC for ParNew: 251 ms for 1 collections, 4900957800 used; = max is 8422162432

Thanks
Bao
--1458549034-1769865613-1368208590=:23005--