From user-return-21964-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Wed Nov 2 14:13:11 2011 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EBF6D9904 for ; Wed, 2 Nov 2011 14:13:10 +0000 (UTC) Received: (qmail 35382 invoked by uid 500); 2 Nov 2011 14:13:08 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 35352 invoked by uid 500); 2 Nov 2011 14:13:08 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 35344 invoked by uid 99); 2 Nov 2011 14:13:08 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Nov 2011 14:13:08 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of s5alye@gmail.com designates 209.85.210.172 as permitted sender) Received: from [209.85.210.172] (HELO mail-iy0-f172.google.com) (209.85.210.172) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 02 Nov 2011 14:13:01 +0000 Received: by iaeo4 with SMTP id o4so215167iae.31 for ; Wed, 02 Nov 2011 07:12:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=XRgcj9N0TYqMU7T6KAUahuZzs0YcTo4gveQkw8XXH1k=; b=jqhWvn2d1ldDf7d+JEeQIA3oO3ax8uen+3k5+JPPmwXbEyC0nRQnchS5R3Vn9yHDCv 8FUxBfV4fPCJa1y9z2g2ndCgsN8nIkqCWiRhlgiZ3eO93u10lZg1sWpvRQIjBKdn10/u 8WwY5DZbPKcZ0/JGqtzOds6mn7bLtcDSpLHQc= MIME-Version: 1.0 Received: by 10.42.151.196 with SMTP id f4mr3497969icw.17.1320243160293; Wed, 02 Nov 2011 07:12:40 -0700 (PDT) Received: by 10.231.199.137 with HTTP; Wed, 2 Nov 2011 07:12:40 -0700 (PDT) Date: Wed, 2 Nov 2011 10:12:40 -0400 Message-ID: Subject: OOM From: A J To: user@cassandra.apache.org Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Checked: Checked by ClamAV on apache.org Hi, For a single node of cassandra(1.0 version) having 15G of data+index, 48GB RAM, 8GB heap and about 2.6G memtable threshold, I am getting OOM when I have 1000 concurrent inserts happening at the same time. I have kept concurrent_writes: 128 in cassandra.yaml as there are a total of 16 cores (suggestion is to keep 8 * number_of_cores). Can someone give pointers on what needs to be tuned. Thanks, AJ. ERROR 00:10:00,312 Fatal exception in thread Thread[Thread-3,5,main] java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:614) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:943) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1336) at org.apache.cassandra.thrift.CustomTThreadPoolServer.serve(CustomTThreadPoolServer.java:105) at org.apache.cassandra.thrift.CassandraDaemon$ThriftServer.run(CassandraDaemon.java:213)