Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1ADE91853B for ; Tue, 23 Feb 2016 14:15:03 +0000 (UTC) Received: (qmail 67445 invoked by uid 500); 23 Feb 2016 14:14:02 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 67403 invoked by uid 500); 23 Feb 2016 14:14:02 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 67393 invoked by uid 99); 23 Feb 2016 14:14:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 23 Feb 2016 14:14:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id EC5BFC06D1 for ; Tue, 23 Feb 2016 14:14:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.18 X-Spam-Level: *** X-Spam-Status: No, score=3.18 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, KAM_BADIPHTTP=2, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id eOpg-wpQLdhW for ; Tue, 23 Feb 2016 14:13:59 +0000 (UTC) Received: from mail-wm0-f45.google.com (mail-wm0-f45.google.com [74.125.82.45]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 4186B5FAFD for ; Tue, 23 Feb 2016 14:13:59 +0000 (UTC) Received: by mail-wm0-f45.google.com with SMTP id g62so224251158wme.1 for ; Tue, 23 Feb 2016 06:13:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:message-id:mime-version:subject:date:references :to:in-reply-to; bh=826NomBv4b1xhckK9PZQKL4Na0ocgANjqcw4jIx1zz0=; b=BsckuUXCvloHmalQY2YoFqKtJottqoTcYtqOzlHxxVdz+5CVcKhCnLEjIQjjqhqi3L fRTSi5wzbw3L4pk6MXBJ1FAD+O/nrKNnJSdpobTAwvQ8YLGVWIoT3ZEl6qm5fn6pE8Ne /khVlPAM134OpJGCSMtTvH5PjATrChLxa6JQv8Sg3O4G1wGjBpB0gbsdTXWkKGKzhouZ sHMPyH5bk4ABZ73Ipxz7huAMSsrYGm612wS/X5mRnImnqUV8Vr7DyPEqLzX566FvZy44 IKfiVva+Qb/zIBJ3+c37qKwMI/B8yh9RdAiqSyi0LuG8aNdQbMnpygqsSequ3rPDYkGX bX1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:content-type:message-id:mime-version :subject:date:references:to:in-reply-to; bh=826NomBv4b1xhckK9PZQKL4Na0ocgANjqcw4jIx1zz0=; b=d63UqMv+Et9Tb9YolDysdoLnqJgGNZ5ddmpc1qwPx+sfYeN+tDdUDU3oApoT2omBr8 OqUlxa+9kypiXhZk845ad+Lo6TMBMv8d4ojptPpG8Owk2Gaf7QuTHHPIQ+A6UwlcbA3q jrvP9cJdBC6Eco/rDyEk9kotl72kbQrf7/yRIGpcIfdvwt5qkn+Q5m57MA8tdfgW6E+d WJyGxyOylKf7i4/I6mD4612t0mPIUmJfuZ1UFXGb6QwGB1wYCiD/IGKJsDyGi6HDzR9P +9GmcdJ/xwFnSxdiFR/NHhpS/1sgxcBAcsGecrxuamDq6z0WH3VBZLUDz8orxxHib/nn qRBQ== X-Gm-Message-State: AG10YORaaGXY7PdYBWPF1EYI6xjDUIJ9pMvEmBy/QCK5kM52aOXApOMIRL+CzbJMtox0Rw== X-Received: by 10.194.92.107 with SMTP id cl11mr39695054wjb.21.1456236839021; Tue, 23 Feb 2016 06:13:59 -0800 (PST) Received: from [10.0.1.41] (syi-fgw2-1498.syi.inet.fi. [88.194.157.218]) by smtp.gmail.com with ESMTPSA id 202sm10560653wmo.7.2016.02.23.06.13.57 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 23 Feb 2016 06:13:57 -0800 (PST) From: =?utf-8?Q?Hannu_Kr=C3=B6ger?= Content-Type: multipart/alternative; boundary="Apple-Mail=_66FEB508-9515-44F7-90D4-0AE0AD64B210" Message-Id: Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\)) Subject: Re: Nodes go down periodically Date: Tue, 23 Feb 2016 16:14:53 +0200 References: To: user@cassandra.apache.org In-Reply-To: X-Mailer: Apple Mail (2.3112) --Apple-Mail=_66FEB508-9515-44F7-90D4-0AE0AD64B210 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi, Those are probably GC pauses. Memory tuning is probably needed. Check = the parameters that you already have customised if they make sense. http://blog.mikiobraun.de/2010/08/cassandra-gc-tuning.html = Hannu > On 23 Feb 2016, at 16:08, Joel Samuelsson = wrote: >=20 > Our nodes go down periodically, around 1-2 times each day. Downtime is = from <1 second to 30 or so seconds. >=20 > INFO [GossipTasks:1] 2016-02-22 10:05:14,896 Gossiper.java (line 992) = InetAddress /109.74.13.67 is now DOWN > INFO [RequestResponseStage:8844] 2016-02-22 10:05:38,331 = Gossiper.java (line 978) InetAddress /109.74.13.67 = is now UP >=20 > I find nothing odd in the logs around the same time. I logged a ping = with timestamp and checked during the same time and saw nothing weird = (ping is less than 2ms at all times). >=20 > Does anyone have any suggestions as to why this might happen? >=20 > Best regards, > Joel --Apple-Mail=_66FEB508-9515-44F7-90D4-0AE0AD64B210 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii Hi,

Those = are probably GC pauses. Memory tuning is probably needed. Check the = parameters that you already have customised if they make = sense.

http://blog.mikiobraun.de/2010/08/cassandra-gc-tuning.html<= /div>

Hannu


On = 23 Feb 2016, at 16:08, Joel Samuelsson <samuelsson.joel@gmail.com> wrote:

Our nodes go down periodically, around 1-2 = times each day. Downtime is from <1 second to 30 or so = seconds.

INFO = [GossipTasks:1] 2016-02-22 10:05:14,896 Gossiper.java (line 992) = InetAddress /109.74.13.67 = is now DOWN
 INFO [RequestResponseStage:8844] = 2016-02-22 10:05:38,331 Gossiper.java (line 978) InetAddress /109.74.13.67 is now = UP

I find = nothing odd in the logs around the same time. I logged a ping with = timestamp and checked during the same time and saw nothing weird (ping = is less than 2ms at all times).

Does anyone have any suggestions as to = why this might happen?

Best regards,
Joel

= --Apple-Mail=_66FEB508-9515-44F7-90D4-0AE0AD64B210--