Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 48C141081C for ; Fri, 26 Jul 2013 08:20:48 +0000 (UTC) Received: (qmail 168 invoked by uid 500); 26 Jul 2013 08:20:45 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 129 invoked by uid 500); 26 Jul 2013 08:20:45 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 121 invoked by uid 99); 26 Jul 2013 08:20:44 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Jul 2013 08:20:44 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of cayiroglu@gmail.com designates 74.125.82.181 as permitted sender) Received: from [74.125.82.181] (HELO mail-we0-f181.google.com) (74.125.82.181) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Jul 2013 08:20:39 +0000 Received: by mail-we0-f181.google.com with SMTP id p58so1551167wes.40 for ; Fri, 26 Jul 2013 01:20:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=NFH5kRml6Y714bbO/jsxXiETJMBOXW1hdvV36dC5Gus=; b=l3SmHIXeeeeUj890txXPERqiqEy7c8jZdW4Op4e+RIVpGltdltnrRrlMOTMosAru/c FbJKUCIAfF28sflAdkcbqkcXc2LyjiHnnXkMLxq9pVRzbZ/OM9WusVcBC5nkNMvVpiZc ZMcgVon40dtSCzoymtLooknqBVAbC9L3kyfjb6mawuBaNi2tQVIoI/LMYUoahjln0ALe CfoRLAzosYyMz6TA8v657A3VWoH3rp9QIRP/kkzweVojv7+hEzchJL3mHLh9W6W9NlZM T7jbPHJcRhIR9vfFti09R2RiYghlAWFL9grxj6yye9WxO6G6RY+XTOl1+rbSCRQjlcWX ubcw== MIME-Version: 1.0 X-Received: by 10.194.122.103 with SMTP id lr7mr33721425wjb.15.1374826818530; Fri, 26 Jul 2013 01:20:18 -0700 (PDT) Received: by 10.194.37.168 with HTTP; Fri, 26 Jul 2013 01:20:18 -0700 (PDT) In-Reply-To: References: <6AA4F6FA5BFDE5459413820369BE16370DB372@AZ-US1EXMB02.global.avaya.com> <6AA4F6FA5BFDE5459413820369BE16370DB38C@AZ-US1EXMB02.global.avaya.com> <57C7C3CBDCB04F45A57AEC4CB21C0CCD1DBE1300@mbx024-e1-nj-6.exch024.domain.local> <6AA4F6FA5BFDE5459413820369BE16370DB3C3@AZ-US1EXMB02.global.avaya.com> Date: Fri, 26 Jul 2013 10:20:18 +0200 Message-ID: Subject: Re: maximum storage per node From: cem To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=089e01175e772575be04e265d013 X-Virus-Checked: Checked by ClamAV on apache.org --089e01175e772575be04e265d013 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable I dont think it is a good idea to put multiple instance in same machine. You may lose multiple instances at the same time if the machine goes down. You can also specify multiple directories as storage in 1.2. I am also not sure boot-strapping will be a big problem since the number keys you will store is relatively small. Why didnt you partition your data according to time instead of using your own compactor? Cem On Fri, Jul 26, 2013 at 3:50 AM, sankalp kohli wrot= e: > Try putting multiple instances per machine with each instance mapped to > its own disk. This might not work with v-nodes > > > On Thu, Jul 25, 2013 at 9:04 AM, Pruner, Anne (Anne) wr= ote: > >> I actually wrote my own compactor that deals with this problem.**** >> >> ** ** >> >> Anne**** >> >> ** ** >> >> *From:* cem [mailto:cayiroglu@gmail.com] >> *Sent:* Thursday, July 25, 2013 11:59 AM >> >> *To:* user@cassandra.apache.org >> *Subject:* Re: maximum storage per node**** >> >> ** ** >> >> You will suffer from long compactions if you are planning to get rid of >> from old records by TTL.**** >> >> ** ** >> >> Best Regards,**** >> >> Cem.**** >> >> ** ** >> >> On Thu, Jul 25, 2013 at 5:51 PM, Kanwar Sangha >> wrote:**** >> >> Issues with large data nodes would be =96**** >> >> **** >> >> =B7 Nodetool repair will be impossible to run**** >> >> =B7 Your read i/o will suffer since you will almost always go to >> disk (each read will take 3 IOPS worst case)**** >> >> =B7 Boot-straping the node in case of failures will take days/we= eks >> **** >> >> **** >> >> **** >> >> *From:* Pruner, Anne (Anne) [mailto:pruner@avaya.com] >> *Sent:* 25 July 2013 10:45 >> *To:* user@cassandra.apache.org >> *Subject:* RE: maximum storage per node**** >> >> **** >> >> We=92re storing fairly large files (about 1MB apiece) for a few months a= nd >> then deleting the oldest to get more space to add new ones. We have lar= ge >> requirements (maybe up to 100 TB), so having a 1TB limit would be >> unworkable.**** >> >> **** >> >> What is the reason for the limit? Does something fail after that?**** >> >> **** >> >> If there are hardware issues, what=92s recommended?**** >> >> **** >> >> BTW, we=92re using Cassandra 1.2**** >> >> **** >> >> Anne**** >> >> **** >> >> *From:* cem [mailto:cayiroglu@gmail.com ] >> *Sent:* Thursday, July 25, 2013 11:41 AM >> *To:* user@cassandra.apache.org >> *Subject:* Re: maximum storage per node**** >> >> **** >> >> Between 500GB - 1TB is recommended. **** >> >> **** >> >> But it depends also your hardware, traffic characteristics and >> requirements. Can you give some details on that?**** >> >> **** >> >> Best Regards,**** >> >> Cem**** >> >> **** >> >> On Thu, Jul 25, 2013 at 5:35 PM, Pruner, Anne (Anne) >> wrote:**** >> >> Does anyone have opinions on the maximum amount of data reasonable to >> store on one Cassandra node? If there are limitations, what are the >> reasons for it?**** >> >> **** >> >> Thanks,**** >> >> Anne**** >> >> **** >> >> ** ** >> > > --089e01175e772575be04e265d013 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
I dont think it is a good idea to put multiple instance in= same machine. You may lose multiple instances at the same time if the mach= ine goes down.=A0You can also specify multiple directories as storage in 1.= 2.=A0

I am also not sure boot-strapping will be a big problem sinc= e the number keys you will store is relatively small.=A0

=
Why didnt you partition your data according to time instead of u= sing your own compactor?

Cem



On Fri, Jul 26, 2013 at 3:50 AM,= sankalp kohli <kohlisankalp@gmail.com> wrote:
Try putting multiple instan= ces per machine with each instance mapped to its own disk. This might not w= ork with v-nodes


=
On Thu, Jul 25, 2013 at 9:04 AM, Pruner, Anne (A= nne) <pruner@avaya.com> wrote:

I actually wrote my own c= ompactor that deals with this problem.

=A0<= /p>

Anne=

=A0<= /p>

From: cem [mai= lto:cayiroglu@gmai= l.com]
Sent: Thursday, July 25, 2013 11:59 AM


To: u= ser@cassandra.apache.org
Subject: Re: maximum storage per node

=A0

You will suffer from long compactions if you are pla= nning to get rid of from old records by TTL.

=A0

Best Regards,

Cem.

=A0

On Thu, Jul 25, 2013 at 5:51 PM, Kanwar Sangha <<= a href=3D"mailto:kanwar@mavenir.com" target=3D"_blank">kanwar@mavenir.com> wrote:

Issues with large data no= des would be =96

=A0<= /p>

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Nodetool repair will be impossible to run=

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Your read i/o will suffer since you will = almost always go to disk (each read will take 3 IOPS worst case)<= /u>

=B7=A0=A0=A0=A0=A0=A0=A0=A0 Boot-straping the node in case of failure= s will take days/weeks

=A0

=A0<= /p>

From: Pruner, = Anne (Anne) [mailto:p= runer@avaya.com]
Sent: 25 July 2013 10:45
To: u= ser@cassandra.apache.org
Subject: RE: maximum storage per node

=A0

We=92re storing fairly la= rge files (about 1MB apiece) for a few months and then deleting the oldest to get more space to add new ones.=A0 We have large requirements (maybe up= to 100 TB), so having a 1TB limit would be unworkable.

=A0<= /p>

What is the reason for th= e limit?=A0 Does something fail after that?

=A0<= /p>

If there are hardware iss= ues, what=92s recommended?

=A0<= /p>

BTW, we=92re using Cassan= dra 1.2

=A0<= /p>

Anne=

=A0<= /p>

From: cem [mailto:cayiroglu@gmai= l.com]
Sent: Thursday, July 25, 2013 11:41 AM
To: u= ser@cassandra.apache.org
Subject: Re: maximum storage per node

=A0

Between 500GB - 1TB is recommended.=A0=

=A0

But it depends also your hardware, traffic character= istics and requirements. Can you give some details on that?

=A0

Best Regards,

Cem

=A0

On Thu, Jul 25, 2013 at 5:35 PM, Pruner, Anne (Anne)= <pruner@avaya.com= > wrote:

Does anyone have opinions= on the maximum amount of data reasonable to store on one Cassandra node?= =A0 If there are limitations, what are the reasons for it?

=A0<= /p>

Thanks,<= /u>

Anne=

=A0

=A0



--089e01175e772575be04e265d013--