Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 053911010A for ; Fri, 26 Jul 2013 01:51:11 +0000 (UTC) Received: (qmail 13820 invoked by uid 500); 26 Jul 2013 01:51:08 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 13802 invoked by uid 500); 26 Jul 2013 01:51:08 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 13793 invoked by uid 99); 26 Jul 2013 01:51:08 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Jul 2013 01:51:08 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of kohlisankalp@gmail.com designates 209.85.216.51 as permitted sender) Received: from [209.85.216.51] (HELO mail-qa0-f51.google.com) (209.85.216.51) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Jul 2013 01:51:02 +0000 Received: by mail-qa0-f51.google.com with SMTP id f11so153928qae.17 for ; Thu, 25 Jul 2013 18:50:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; bh=p4b+HI3U013KPHuy44Uxkrkn7pRJpDvMl0xCdJ2Hme8=; b=svQNRNuD1ZO9uaQ8dtnES196XJASZoIUfh+Noui1SoXPw/8RxewDVNN8H2zqxXMBU8 frFUF1x4vegpKCBRkXRMmQEoCIl3YBP4d/41qYQgzJ38ajFDU8EtkSrxu9yOyOsaqfgN TcaxWqpFzd9Q+hKY5/sahIg4ajhZwvZSDZmErLg+l5ztRiQk4SuIayAxJpVAA8kOT370 j6GLuxZAuH3o9sAhfcAWwsnFZer1uFs5QotC6A3btLN8fCT2tLbGoGcjtg9t0aRTkvKD 6ZxY4l0U699b55F/PUXBygW3Vkg7NbDTitSrzcySu3Y7p8AFxOUncVnugagsE0fpNY2+ wjYg== X-Received: by 10.224.136.72 with SMTP id q8mr1867648qat.35.1374803441282; Thu, 25 Jul 2013 18:50:41 -0700 (PDT) MIME-Version: 1.0 Received: by 10.49.0.171 with HTTP; Thu, 25 Jul 2013 18:50:01 -0700 (PDT) In-Reply-To: <6AA4F6FA5BFDE5459413820369BE16370DB3C3@AZ-US1EXMB02.global.avaya.com> References: <6AA4F6FA5BFDE5459413820369BE16370DB372@AZ-US1EXMB02.global.avaya.com> <6AA4F6FA5BFDE5459413820369BE16370DB38C@AZ-US1EXMB02.global.avaya.com> <57C7C3CBDCB04F45A57AEC4CB21C0CCD1DBE1300@mbx024-e1-nj-6.exch024.domain.local> <6AA4F6FA5BFDE5459413820369BE16370DB3C3@AZ-US1EXMB02.global.avaya.com> From: sankalp kohli Date: Thu, 25 Jul 2013 18:50:01 -0700 Message-ID: Subject: Re: maximum storage per node To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=001a11c2b950c0f36604e2605e7f X-Virus-Checked: Checked by ClamAV on apache.org --001a11c2b950c0f36604e2605e7f Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Try putting multiple instances per machine with each instance mapped to its own disk. This might not work with v-nodes On Thu, Jul 25, 2013 at 9:04 AM, Pruner, Anne (Anne) wrot= e: > I actually wrote my own compactor that deals with this problem.**** > > ** ** > > Anne**** > > ** ** > > *From:* cem [mailto:cayiroglu@gmail.com] > *Sent:* Thursday, July 25, 2013 11:59 AM > > *To:* user@cassandra.apache.org > *Subject:* Re: maximum storage per node**** > > ** ** > > You will suffer from long compactions if you are planning to get rid of > from old records by TTL.**** > > ** ** > > Best Regards,**** > > Cem.**** > > ** ** > > On Thu, Jul 25, 2013 at 5:51 PM, Kanwar Sangha wrote= : > **** > > Issues with large data nodes would be =96**** > > **** > > =B7 Nodetool repair will be impossible to run**** > > =B7 Your read i/o will suffer since you will almost always go to > disk (each read will take 3 IOPS worst case)**** > > =B7 Boot-straping the node in case of failures will take days/wee= ks* > *** > > **** > > **** > > *From:* Pruner, Anne (Anne) [mailto:pruner@avaya.com] > *Sent:* 25 July 2013 10:45 > *To:* user@cassandra.apache.org > *Subject:* RE: maximum storage per node**** > > **** > > We=92re storing fairly large files (about 1MB apiece) for a few months an= d > then deleting the oldest to get more space to add new ones. We have larg= e > requirements (maybe up to 100 TB), so having a 1TB limit would be > unworkable.**** > > **** > > What is the reason for the limit? Does something fail after that?**** > > **** > > If there are hardware issues, what=92s recommended?**** > > **** > > BTW, we=92re using Cassandra 1.2**** > > **** > > Anne**** > > **** > > *From:* cem [mailto:cayiroglu@gmail.com ] > *Sent:* Thursday, July 25, 2013 11:41 AM > *To:* user@cassandra.apache.org > *Subject:* Re: maximum storage per node**** > > **** > > Between 500GB - 1TB is recommended. **** > > **** > > But it depends also your hardware, traffic characteristics and > requirements. Can you give some details on that?**** > > **** > > Best Regards,**** > > Cem**** > > **** > > On Thu, Jul 25, 2013 at 5:35 PM, Pruner, Anne (Anne) > wrote:**** > > Does anyone have opinions on the maximum amount of data reasonable to > store on one Cassandra node? If there are limitations, what are the > reasons for it?**** > > **** > > Thanks,**** > > Anne**** > > **** > > ** ** > --001a11c2b950c0f36604e2605e7f Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
Try putting multiple instances per machine with each insta= nce mapped to its own disk. This might not work with v-nodes


On Thu, Jul 25, 2013 a= t 9:04 AM, Pruner, Anne (Anne) <pruner@avaya.com> wrote:

I actually wrote my own c= ompactor that deals with this problem.

=A0<= /p>

Anne=

=A0<= /p>

From: cem [mai= lto:cayiroglu@gmai= l.com]
Sent: Thursday, July 25, 2013 11:59 AM


To: u= ser@cassandra.apache.org
Subject: Re: maximum storage per node

=A0

You will suffer from long compactions if you are pla= nning to get rid of from old records by TTL.

=A0

Best Regards,

Cem.

=A0

On Thu, Jul 25, 2013 at 5:51 PM, Kanwar Sangha <<= a href=3D"mailto:kanwar@mavenir.com" target=3D"_blank">kanwar@mavenir.com> wrote:

Issues with large data no= des would be =96

=A0<= /p>

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Nodetool repair will be impossible to run=

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Your read i/o will suffer since you will = almost always go to disk (each read will take 3 IOPS worst case)<= /u>

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Boot-straping the node in case of failure= s will take days/weeks

=A0

=A0<= /p>

From: Pruner, = Anne (Anne) [mailto:p= runer@avaya.com]
Sent: 25 July 2013 10:45
To: u= ser@cassandra.apache.org
Subject: RE: maximum storage per node

=A0

We=92re storing fairly la= rge files (about 1MB apiece) for a few months and then deleting the oldest to get more space to add new ones.=A0 We have large requirements (maybe up= to 100 TB), so having a 1TB limit would be unworkable.

=A0<= /p>

What is the reason for th= e limit?=A0 Does something fail after that?

=A0<= /p>

If there are hardware iss= ues, what=92s recommended?

=A0<= /p>

BTW, we=92re using Cassan= dra 1.2

=A0<= /p>

Anne=

=A0<= /p>

From: cem [mailto:cayiroglu@gmai= l.com]
Sent: Thursday, July 25, 2013 11:41 AM
To: u= ser@cassandra.apache.org
Subject: Re: maximum storage per node

=A0

Between 500GB - 1TB is recommended.=A0=

=A0

But it depends also your hardware, traffic character= istics and requirements. Can you give some details on that?

=A0

Best Regards,

Cem

=A0

On Thu, Jul 25, 2013 at 5:35 PM, Pruner, Anne (Anne)= <pruner@avaya.com= > wrote:

Does anyone have opinions= on the maximum amount of data reasonable to store on one Cassandra node?= =A0 If there are limitations, what are the reasons for it?

=A0<= /p>

Thanks,<= /u>

Anne=

=A0

=A0


--001a11c2b950c0f36604e2605e7f--