Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 827D010895 for ; Mon, 15 Apr 2013 13:25:37 +0000 (UTC) Received: (qmail 20795 invoked by uid 500); 15 Apr 2013 13:25:35 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 20485 invoked by uid 500); 15 Apr 2013 13:25:34 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 20438 invoked by uid 99); 15 Apr 2013 13:25:33 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 15 Apr 2013 13:25:33 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.217.178] (HELO mail-lb0-f178.google.com) (209.85.217.178) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 15 Apr 2013 13:25:29 +0000 Received: by mail-lb0-f178.google.com with SMTP id q13so4530253lbi.9 for ; Mon, 15 Apr 2013 06:25:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:content-type :x-gm-message-state; bh=UyKnrSphz2ppm99Y/42selyJRJqFnrgMtFk1MePHuUc=; b=hUnPDTTO3DrXmedG2UdevWWQmoIt6PBxBe5YsJZ0Eom3o5v0ownjLY8yXWLzwwOzst Ae7Oe7X5fde3n1qPyOr5+OA82eN4TdXyOFaebAMp9FHandkjoAIBctEbyh0dA0RaaCIp y2dKF0kVeVdqTLDRDRls92qPW4j8XDJol4bsCliai7UF/j4+zURajBsWF2OHQcIG2e02 E9SwOLBh0oAuLG3Y5mQ0MXhYe2Xgg1lU/+Ie5abIU+ekqsPtrWuDrf+bm5Ykzg1yrcOO tXk1pKTQXq9jwr4V6u11bYP2+ILqN4VVR5QnK8bFAbhlwgZ5ANsUvoYnylARCuxGavu9 8t+w== X-Received: by 10.152.6.102 with SMTP id z6mr10444376laz.7.1366032307627; Mon, 15 Apr 2013 06:25:07 -0700 (PDT) MIME-Version: 1.0 Sender: niki@photonhost.com Received: by 10.114.186.198 with HTTP; Mon, 15 Apr 2013 06:24:37 -0700 (PDT) In-Reply-To: References: From: Nikolay Mihaylov Date: Mon, 15 Apr 2013 16:24:37 +0300 X-Google-Sender-Auth: uyq3l4u1RrT_zRPwc1Ol7vjlmcQ Message-ID: Subject: Re: running cassandra on 8 GB servers To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=089e0149401c72bec204da662e30 X-Gm-Message-State: ALoCoQkVupdM5XWytGSDLfifWCvd9Rj6sPQiF+HCbCuem2LU+0OTft64EdGsxCgxc9Wop619EucD X-Virus-Checked: Checked by ClamAV on apache.org --089e0149401c72bec204da662e30 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Just a small update here currently running on one node with 7 GB heap and no JNA all defaults except the heap, and everything looks OK. On Sun, Apr 14, 2013 at 9:10 PM, aaron morton wrot= e: > Hmmm, what is the recommendation for a 10G network if 1G was 300G to > 500G=C5=A0I am guessing I can't do 10 times that, correct? But maybe I c= ould > squeak out 600G to 1T? > > Best thing to do would be run a test on how long it takes to repair or > bootstrap a node. The 300GB to 500Gb was just a guideline. > > Cheers > > ----------------- > Aaron Morton > Freelance Cassandra Consultant > New Zealand > > @aaronmorton > http://www.thelastpickle.com > > On 13/04/2013, at 12:02 AM, "Hiller, Dean" wrote: > > Hmmm, what is the recommendation for a 10G network if 1G was 300G to > 500G=C5=A0I am guessing I can't do 10 times that, correct? But maybe I c= ould > squeak out 600G to 1T? > > Thanks, > Dean > > On 4/11/13 2:26 PM, "aaron morton" wrote: > > The data will be huge, I am estimating 4-6 TB per server. I know this > is best, but those are my resources. > > You will have a very unhappy time. > > The general rule of thumb / guideline for a HDD based system with 1G > networking is 300GB to 500Gb per node. See previous discussions on this > topic for reasons. > > ERROR [Thrift:641] 2013-04-11 11:25:19,563 CassandraDaemon.java (line > 164) Exception in thread Thread[Thrift:641,5,main] > ... > INFO [StorageServiceShutdownHook] 2013-04-11 11:25:39,915 > ThriftServer.java (line 116) Stop listening to thrift clients > > What was the error ? > > What version are you using? > If you have changed any defaults for memory in cassandra-env.sh or > cassandra.yaml revert them. Generally C* will do the right thing and not > OOM, unless you are trying to store a lot of data on a node that does not > have enough memory. See this thread for background > http://www.mail-archive.com/user@cassandra.apache.org/msg25762.html > > Cheers > > ----------------- > Aaron Morton > Freelance Cassandra Consultant > New Zealand > > @aaronmorton > http://www.thelastpickle.com > > On 12/04/2013, at 7:35 AM, Nikolay Mihaylov wrote: > > For one project I will need to run cassandra on following dedicated > servers: > > Single CPU XEON 4 cores no hyper-threading, 8 GB RAM, 12 TB locally > attached HDD's in some kind of RAID, visible as single HDD. > > I can do cluster of 20-30 such servers, may be even more. > > The data will be huge, I am estimating 4-6 TB per server. I know this > is best, but those are my resources. > > Currently I am testing with one of such servers, except HDD is 300 GB. > Every 15-20 hours, I get out of heap memory, e.g. something like: > > ERROR [Thrift:641] 2013-04-11 11:25:19,563 CassandraDaemon.java (line > 164) Exception in thread Thread[Thrift:641,5,main] > ... > INFO [StorageServiceShutdownHook] 2013-04-11 11:25:39,915 > ThriftServer.java (line 116) Stop listening to thrift clients > INFO [StorageServiceShutdownHook] 2013-04-11 11:25:39,943 > Gossiper.java (line 1077) Announcing shutdown > INFO [StorageServiceShutdownHook] 2013-04-11 11:26:08,613 > MessagingService.java (line 682) Waiting for messaging service to quiesce > INFO [ACCEPT-/208.94.232.37] 2013-04-11 11:26:08,655 > MessagingService.java (line 888) MessagingService shutting down server > thread. > ERROR [Thrift:721] 2013-04-11 11:26:37,709 CustomTThreadPoolServer.java > (line 217) Error occurred during processing of message. > java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has > shut down > > Anyone have some advices about better utilization of such servers? > > Nick. > > > > > --089e0149401c72bec204da662e30 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Just a small update here
currently running on one node with 7 GB heap an= d no JNA
all defaults except the heap, and everything looks OK.

On Sun, Apr 14, 2013 at 9:10 PM, aaron mo= rton <aaron@thelastpickle.com> wrote:
Hmmm, what is the recommendation for a = 10G network if 1G was 300G to
500G=C5=A0I am guessing I can't do 10 times that, correct? =C2=A0But ma= ybe I could
squeak out 600G to 1T?
Best thing to do wo= uld be run a test on how long it takes to repair or bootstrap a node. The 3= 00GB to 500Gb was just a guideline.

Cheers

-----------------
Aaron Morton
Freelance Cassandra= Consultant
New Zealand


On 13/04/2013, at 12:02 AM, &quo= t;Hiller, Dean" <Dean.Hiller@nrel.gov> wrote:

Hmmm, what is the recommendation for a 10G network if 1G was 300G to
500= G=C5=A0I am guessing I can't do 10 times that, correct? =C2=A0But maybe= I could
squeak out 600G to 1T?

Thanks,
Dean

On 4/11/13= 2:26 PM, "aaron morton" <aaron@thelastpickle.com> wrote:

The data will be hu= ge, I am estimating 4-6 TB per server. I know this
is best, but those ar= e my resources.
You will have a very unhappy time.

The general rule of thumb / guideline for a HDD based system with 1G
networking is 300GB to 500Gb per node. See previous discussions on this
= topic for reasons.

ERROR [Thrift:641] 2013= -04-11 11:25:19,563 CassandraDaemon.java (line
164) Exception in thread = Thread[Thrift:641,5,main]
...
INFO [StorageServiceShutdownHook] 2013-04-11 11:25:39,915
Thrift= Server.java (line 116) Stop listening to thrift clients
Wha= t was the error ?

What version are you using?
If you have changed= any defaults for memory in cassandra-env.sh or
cassandra.yaml revert them. Generally C* will do the right thing and notOOM, unless you are trying to store a lot of data on a node that does not<= br>have enough memory. See this thread for background
http://www.mail-archive.com/user@cassandra.apache.org/msg25762.html

Cheers

-----------------
Aaron Morton
Freelance Cassandra = Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On= 12/04/2013, at 7:35 AM, Nikolay Mihaylov <nmmm@nmmm.nu> wrote:

For one project I will need to run cassandra = on following dedicated
servers:

Single CPU XEON 4 cores no hyper-= threading, 8 GB RAM, 12 TB locally
attached HDD's in some kind of RA= ID, visible as single HDD.

I can do cluster of 20-30 such servers, may be even more.

The da= ta will be huge, I am estimating 4-6 TB per server. I know this
is best,= but those are my resources.

Currently I am testing with one of such= servers, except HDD is 300 GB.
Every 15-20 hours, I get out of heap memory, e.g. something like:

ER= ROR [Thrift:641] 2013-04-11 11:25:19,563 CassandraDaemon.java (line
164)= Exception in thread Thread[Thrift:641,5,main]
...
INFO [StorageServ= iceShutdownHook] 2013-04-11 11:25:39,915
ThriftServer.java (line 116) Stop listening to thrift clients
INFO [Sto= rageServiceShutdownHook] 2013-04-11 11:25:39,943
Gossiper.java (line 107= 7) Announcing shutdown
INFO [StorageServiceShutdownHook] 2013-04-11 11:= 26:08,613
MessagingService.java (line 682) Waiting for messaging service to quiesce INFO [ACCEPT-/208.94.= 232.37] 2013-04-11 11:26:08,655
MessagingService.java (line 888) Mes= sagingService shutting down server
thread.
ERROR [Thrift:721] 2013-04-11 11:26:37,709 CustomTThreadPoolServ= er.java
(line 217) Error occurred during processing of message.
java.= util.concurrent.RejectedExecutionException: ThreadPoolExecutor has
shut down

Anyone have some advices about better utilization of such servers?
<= br>Nick.



=

--089e0149401c72bec204da662e30--