From user-return-60465-archive-asf-public=cust-asf.ponee.io@cassandra.apache.org Tue Mar 20 14:23:30 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 418DE18064A for ; Tue, 20 Mar 2018 14:23:30 +0100 (CET) Received: (qmail 87356 invoked by uid 500); 20 Mar 2018 13:23:28 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 87346 invoked by uid 99); 20 Mar 2018 13:23:28 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 20 Mar 2018 13:23:28 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id DB55D1A01D5 for ; Tue, 20 Mar 2018 13:23:27 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.898 X-Spam-Level: * X-Spam-Status: No, score=1.898 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id i81uAzd6_eGs for ; Tue, 20 Mar 2018 13:23:26 +0000 (UTC) Received: from mail-vk0-f44.google.com (mail-vk0-f44.google.com [209.85.213.44]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 32AC55F5DD for ; Tue, 20 Mar 2018 13:23:25 +0000 (UTC) Received: by mail-vk0-f44.google.com with SMTP id k187so888781vke.12 for ; Tue, 20 Mar 2018 06:23:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=O9I6VE/F54b2u02OPXVMznGPkpXic2hrC52ORhyjbyE=; b=N3MLNzhys6Zf7sf/wu6jS3CrBV8kDc5HrdnVXp9BsTvCW6oK4FxA5V+Q8zkKW7wHPq BK4cn0Owa7wv5v0WTPJYEYDj2o9Jnzsspb7olsgCuIg2UHTzcM7T9ZtNuWnhTpeDo+1f eDTFEgIlKeogwi2rDvdmxOaJHzEv3Omo9CdyKS5HZMiQXNOgJRyvyPxJYNKDCEHoMfoi LnufNi2d6moXkSVLLEyzqnLztdyGVZufYN8cPbD6sZ0293ww7MGZnUiatyIfx/2yTYVO YwGO0DltfB/KjQycQNrHVyw7Bu+cj+UIT7rMPLo99gjCKSeboXZm2f9jVpf2iOQOaWAM c4Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=O9I6VE/F54b2u02OPXVMznGPkpXic2hrC52ORhyjbyE=; b=RPzwxd5j+oLB8bIRX6HWJi3wB0Qa4BT/CmPByukPsUI5vFeENVUbduaNM8Dr/UPYPd YlLa/aimZ+P64EoxVrHKJDh7Herah/uONsnPL5t42VyZUIQcq7BMAqSnRzurhGCnstjJ y/Drk9ra3G9LKVBJ4Oqb4M2oxGULinfbconPqv3UnvHd3pCwwCx3nzuVYd/1rinI0AWv 7KRi55aEM2HxqRBv3yF34Ur2TcEYGeRnQKRNjF/LhX+b0ZFTeJXVpLY1A+ukNomP/PFK Zqk/TS4BzECyexHbWtmq6jA4/sA84m3FFC+quZV3KiEs/vTx1MLoCrWk0ONMXVxjFTT2 80kQ== X-Gm-Message-State: AElRT7F832nZ7/+BJfq+w5BssNSSsN0S62POlqsU/ABjiHSCIuCjf37N UgP+P0sw2LJSLuH6KfKnX0MapyF/ALe/rwnUzxbQ0vTb X-Google-Smtp-Source: AG47ELt0ZNEq6DUatep8GmagtM7PNu5tacMR0fLXfxUWp5EjND3ykdjIkXjB81aPcs+Gp1WhzbAcQ59aeRNGoNjSzj0= X-Received: by 10.31.175.129 with SMTP id y123mr10157275vke.189.1521552203544; Tue, 20 Mar 2018 06:23:23 -0700 (PDT) MIME-Version: 1.0 Received: by 10.103.131.65 with HTTP; Tue, 20 Mar 2018 06:23:02 -0700 (PDT) In-Reply-To: References: From: Alain RODRIGUEZ Date: Tue, 20 Mar 2018 13:23:02 +0000 Message-ID: Subject: Re: Anomaly detection To: "user cassandra.apache.org" Content-Type: multipart/alternative; boundary="001a1144140e997c550567d7fa70" --001a1144140e997c550567d7fa70 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello Salvatore, I used SPM Sematext that had this feature already 3 or 4 years ago I believe. SPM offers some out of the box dashboards for Cassandra. See https://sematext.com/spm/ More recently we built some templates for Cassandra in Data Dog that I believe should show the most important metrics and hopefully in a usable/efficient way for alerting and troubleshooting. If this is of interest, check https://www.datadoghq.com/blog/tlp-cassandra-dashboards/. Datadog provides anomaly detection for a while too: https://www.datadoghq.com/blog/introducing-anomaly-detection-datadog/. If you choose Datadog, adding anomaly detection to the overview dashboard charts is probably a good starting point. The tools mentioned above are commercial solutions. I am not sure about which open source stacks could provide this feature. That being said, looking from time to time to the dashboards (at least) and setting alerts using the standard 'thresholds' was often enough. I did not make real use of anomaly detections, even though there are probably interesting things that can be done on alerting with this feature. Hope that helps, C*heers, ----------------------- Alain Rodriguez - @arodream - alain@thelastpickle.com France / Spain The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com 2018-03-13 10:07 GMT+00:00 Rahul Singh : > I=E2=80=99ve used OpsCenter, New Relic, Splunk, and ELK and all of them h= ave ways > to visualize what=E2=80=99s going on. Eventually I just forked a cfstats2= csv python > program and started making formatted excel files which made it easy to sp= ot > anomalies and filter keyspaces / tables across nodes. I have some basic > anomaly detection based on std. deviation but it=E2=80=99s only a static = snapshot > based detection. I think you want something that may be looking at the ti= me > series of the whole dataset. > > Regardless cfstats , tpstats are good places to see > What=E2=80=99s going on and then determine what you need to monitor via o= ther > tools. > > -- > Rahul Singh > rahul.singh@anant.us > > Anant Corporation > > On Mar 12, 2018, 10:02 PM -0400, Fernando Ipar , > wrote: > > Hello Salvatore, > > On Mon, Mar 12, 2018 at 2:12 PM, D. Salvatore > wrote: > >> Hi Rahul, >> I was mainly thinking about performance anomaly detection but I am also >> interested in other types such as fault detection, data or queries >> anomalies. >> > > I know VividCortex (http://vividcortex.com) supports Cassandra (2.1 or > greater) and I also know it does automatic (they call it adaptive) fault > detection for MySQL. I took a quick look at their website and could not > find an explicit list of features they support for Cassandra but it's > possible that fault detection is one of them too, so if SaaS is an option > I'd recommend you take a look at them. > > Regards, > > Fernando Ipar > > http://fernandoipar.com > > --001a1144140e997c550567d7fa70 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello Salvatore,

I used SPM Sematext=C2= =A0that had this feature already 3 or 4 years ago=C2=A0I believe. SPM offer= s some out of the box dashboards for Cassandra. See=C2=A0https://sematext.com/spm/

More recently we b= uilt some templates for Cassandra in Data Dog that I believe should show th= e most important metrics and hopefully in a usable/efficient way for alerti= ng and troubleshooting. If this is of interest, check https://www.datadoghq.com/b= log/tlp-cassandra-dashboards/. Datadog provides anomaly detection for a= while too:=C2=A0https://www.datadoghq.com/blog/introducing-anomaly-= detection-datadog/.

If you choose Datadog, add= ing anomaly detection to the overview dashboard charts is probably a good s= tarting point.

The tools mentioned above are commercial solutions. I= am not sure about which open source stacks could provide this feature.
=
That being said, looking from time to time to the dashboards (at least)= and setting alerts using the standard 'thresholds' was often enoug= h. I did not make real use of anomaly detections, even though there are pro= bably interesting things that can be done on alerting with this feature.

Hope that helps,

C*heers,
-----------------------
Alain Rodriguez - @arodream= - alain@thelastpickle.com
France / Spain

The Last Pickle - Apache C= assandra Consulting



2018-03-13 10:07 GMT+00:00 Rahul Singh= <rahul.xavier.singh@gmail.com>:
I=E2=80=99ve used OpsCenter, New Relic, Sp= lunk, and ELK and all of them have ways to visualize what=E2=80=99s going o= n. Eventually I just forked a cfstats2csv python program and started making= formatted excel files which made it easy to spot anomalies and filter keys= paces / tables across nodes. I have some basic anomaly detection based on s= td. deviation but it=E2=80=99s only a static snapshot based detection. I th= ink you want something that may be looking at the time series of the whole = dataset.

Regardless cfstats , tpstats are good places to see
What=E2=80=99s going on and then determine what you need to monitor via oth= er tools.

--
Rahul Singh
rahul.singh@anant= .us

Anant Corporation

On Mar 12, 2018, 10:02 PM -0400, Fernando Ipar <ipar.fernando@gmail.com>, wrote= :
Hello Salvatore,=C2=A0

On Mon, Mar 12, 2018 at 2:12 PM, D. Salvatore <dd.salvatore@gmail.com> wrote:
Hi Rahul,
I was mainly thinking about performance anomaly detection but I am als= o interested in other types such as fault detection, data or queries anomal= ies.

I know VividCortex (http://vividcortex.com) supports Cassandra (2.1 or greater) and I al= so know it does automatic (they call it adaptive) fault detection for MySQL= . I took a quick look at their website and could not find an explicit list = of features they support for Cassandra but it's possible that fault det= ection is one of them too, so if SaaS is an option I'd recommend you ta= ke a look at them.=C2=A0

Regards,=C2=A0

Fernando Ipar

http://fernandoip= ar.com=C2=A0


--001a1144140e997c550567d7fa70--