Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CB09F192D0 for ; Thu, 14 Apr 2016 11:48:10 +0000 (UTC) Received: (qmail 1005 invoked by uid 500); 14 Apr 2016 11:48:04 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 957 invoked by uid 500); 14 Apr 2016 11:48:04 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 947 invoked by uid 99); 14 Apr 2016 11:48:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 14 Apr 2016 11:48:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 1F799C08DA for ; Thu, 14 Apr 2016 11:48:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.179 X-Spam-Level: * X-Spam-Status: No, score=1.179 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id YhbODMV6PCn6 for ; Thu, 14 Apr 2016 11:48:02 +0000 (UTC) Received: from mail-vk0-f45.google.com (mail-vk0-f45.google.com [209.85.213.45]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 276D25F23E for ; Thu, 14 Apr 2016 11:48:01 +0000 (UTC) Received: by mail-vk0-f45.google.com with SMTP id e185so105841112vkb.1 for ; Thu, 14 Apr 2016 04:48:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=WtFSf4UOjaXY+xAMy3wk20CwUULx3YDFdbXbtDhBSFA=; b=Xioo7aCZzgAoCsg1Tn4fSZCWCJjIMXvNroXGThjkk2jSswRXZya74aRfygYBVsRnB2 MCQbSMAA+8ZFOGf3PsrdcY8JzFp6BrWMExF5IPrtzIvATEz3zSWTgUR2C3+bvPdFFHjO Sx3c/RGjNzMRqotKVdDy6ctmBDML5TL9GY92wTW87eKA0AEHHxhGCnHQwj5qRETFIpRu GIw+nifdaD1NqbGX10Wx4qgwYuSqj0jTZYko608rveFTDCK8ZZ6/1JQDUGpXibczPSeI SS0ZKNagksfFKH4VxfrFLxCS60oC5bisaQtx2b42+OmIBNZnm34vynrBIeNVYV2cwNu2 sQMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=WtFSf4UOjaXY+xAMy3wk20CwUULx3YDFdbXbtDhBSFA=; b=TINd59HwXrr917OC3J4HCRzANwwctOURb/h452Ks+Kkt2Xu0uW6Y6Yo0sQcKIQ0h7X 65sWgEGo1X3F2zp90SDQgaq1jgu4Oc3bDGY0YsiE2zCarQUZ0vTdQ69K/2BcGHNpwa09 OZp6pf1B37BLmgAbfAub8KoYulVGJHK+uXI+Q9ZovW6oqk1y3nZUiAYSp+Wu+av3LuKf FNcAVzNVxXiI4pMqRCMdAKkb/1wPKP66/uw/DIPUqqEmUvBStf3XahhBhAHlsYYCOwrw N+F04vQ34/pay24ksRzrk8LdkXqJRZtvnFjbWU9O3GgaUyVif3nFpmr/kPoU756hgKx9 m0Zg== X-Gm-Message-State: AOPr4FWRi5icdexRC0o1uedLUB8RgnJPx+19vASXbeBgn5r3t8qE8Hq4o6AaFh+dR+J0y2Y0lqzw4JPlcJCKhA== X-Received: by 10.31.142.203 with SMTP id q194mr7257099vkd.95.1460634480218; Thu, 14 Apr 2016 04:48:00 -0700 (PDT) MIME-Version: 1.0 Received: by 10.159.32.11 with HTTP; Thu, 14 Apr 2016 04:47:40 -0700 (PDT) In-Reply-To: References: From: Alain RODRIGUEZ Date: Thu, 14 Apr 2016 13:47:40 +0200 Message-ID: Subject: Re: Cassandra 2.1.12 Node size To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=001a1143a728571704053070774c --001a1143a728571704053070774c Content-Type: text/plain; charset=UTF-8 Hi, I seek advice in data size per node. Each of my node has close to 1 TB of > data. I am not seeing any issues as of now but wanted to run it by you guys > if this data size is pushing the limits in any manner and if I should be > working on reducing data size per node. There is no real limit to the data size other than 50% of the machine disk space using STCS and 80 % if you are using LCS. Those are 'soft' limits as it will depend on your biggest sstables size and the number of concurrent compactions mainly, but to stay away from trouble, it is better to keep things under control, below the limits mentioned above. I will me migrating to incremental repairs shortly and full repair as of > now takes 20 hr/node. I am not seeing any issues with the nodes for now. > As you noticed, you need to keep in mind that the larger the dataset is, the longer operations will take. Repairs but also bootstrap or replace a node, remove a node, any operation that require to stream data or read it. Repair time can be mitigated by using incremental repairs indeed. I am running a 9 node C* 2.1.12 cluster. > It should be quite safe to give incremental repair a try as many bugs have been fixe in this version: FIX 2.1.12 - A lot of sstables using range repairs due to anticompaction - incremental only https://issues.apache.org/jira/browse/CASSANDRA-10422 FIX 2.1.12 - repair hang when replica is down - incremental only https://issues.apache.org/jira/browse/CASSANDRA-10288 If you are using DTCS be aware of https://issues.apache.org/jira/browse/CASSANDRA-11113 If using LCS, watch closely sstable and compactions pending counts. As a general comment, I would say that Cassandra has evolved to be able to handle huge datasets (memory structures off-heap + increase of heap size using G1GC, JBOD, vnodes, ...). Today Cassandra works just fine with big dataset. I have seen clusters with 4+ TB nodes and other using a few GB per node. It all depends on your requirements and your machines spec. If fast operations are absolutely necessary, keep it small. If you want to use the entire disk space (50/80% of total disk space max), go ahead as long as other resources are fine (CPU, memory, disk throughput, ...). C*heers, ----------------------- Alain Rodriguez - alain@thelastpickle.com France The Last Pickle - Apache Cassandra Consulting http://www.thelastpickle.com 2016-04-14 10:57 GMT+02:00 Aiman Parvaiz : > Hi all, > I am running a 9 node C* 2.1.12 cluster. I seek advice in data size per > node. Each of my node has close to 1 TB of data. I am not seeing any issues > as of now but wanted to run it by you guys if this data size is pushing the > limits in any manner and if I should be working on reducing data size per > node. I will me migrating to incremental repairs shortly and full repair as > of now takes 20 hr/node. I am not seeing any issues with the nodes for now. > > Thanks > > > > --001a1143a728571704053070774c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi,
I seek advice in data size = per node. Each of my node has close to 1 TB of data. I am not seeing any is= sues as of now but wanted to run it by you guys if this data size is pushin= g the limits in any manner and if I should be working on reducing data size= per node.

There= is no real limit to the data size other than 50% of the machine disk space= using STCS and 80 % if you are using LCS. Those are 'soft' limits = as it will depend on your biggest sstables size and the number of concurren= t compactions mainly, but to stay away from trouble, it is better to keep t= hings under control, below the limits mentioned above.

= I will me migrating to incremen= tal repairs shortly and full repair as of now takes 20 hr/node. I am not se= eing any issues with the nodes for now.

As you noticed, you need to keep in mind that t= he larger the dataset is, the longer operations will take. Repairs but also=C2=A0= bootstrap or replace= a node, remove a node, any operation that require to stream data or read i= t. Repair time can be mitigated by using incremental repairs indeed.=C2=A0<= /span>

I am runnin= g a 9 node C* 2.1.12 cluster.

It should be quite safe to give incremental repair a try = as many bugs have been fixe in this version:

FIX 2.1.12 - A lot of = sstables using range repairs due to anticompaction -=C2=A0incremental only<= /font>

https://issue= s.apache.org/jira/browse/CASSANDRA-10422

FIX 2.1.12 - repair ha= ng when replica is down - incremental only

https://issue= s.apache.org/jira/browse/CASSANDRA-10288

If you are using DTCS be aware of https://issues.apache.org/jira/browse/CASSANDRA-11113

If using LCS, watch closely sstab= le and compactions pending counts.

As a general comment, I would say that Cassandra has evolv= ed to be able to handle huge datasets (memory structures off-heap + increas= e of heap size using G1GC, JBOD, vnodes, ...). Today Cassandra works just f= ine with big dataset. I have seen clusters with 4+ TB nodes and other using= a few GB per node. It all depends on your requirements and your machines s= pec. If fast operations are absolutely necessary, keep it small. If you wan= t to use the entire disk space (50/80% of total disk space max), go ahead a= s long as other resources are fine (CPU, memory, disk throughput, ...).

C*heers,

-----------------------
Alain Rodriguez = - alain@thelas= tpickle.com
France

The L= ast Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com

2016-04-14 10:57 GMT+02:00 Aiman Parvaiz <aiman@flipagram.com>:

--001a1143a728571704053070774c--