Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 44269 invoked from network); 18 Sep 2010 17:21:28 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 18 Sep 2010 17:21:28 -0000 Received: (qmail 77823 invoked by uid 500); 18 Sep 2010 17:21:27 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 77778 invoked by uid 500); 18 Sep 2010 17:21:26 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 77770 invoked by uid 99); 18 Sep 2010 17:21:26 -0000 Received: from Unknown (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 18 Sep 2010 17:21:26 +0000 X-ASF-Spam-Status: No, hits=0.7 required=10.0 tests=RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.213.44] (HELO mail-yw0-f44.google.com) (209.85.213.44) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 18 Sep 2010 17:21:01 +0000 Received: by ywk9 with SMTP id 9so1394672ywk.31 for ; Sat, 18 Sep 2010 10:20:37 -0700 (PDT) MIME-Version: 1.0 Received: by 10.150.167.36 with SMTP id p36mr6905342ybe.413.1284830437130; Sat, 18 Sep 2010 10:20:37 -0700 (PDT) Sender: scode@scode.org Received: by 10.150.138.16 with HTTP; Sat, 18 Sep 2010 10:20:36 -0700 (PDT) X-Originating-IP: [213.114.156.79] In-Reply-To: References: <4C940305.7070905@pdf.com> Date: Sat, 18 Sep 2010 19:20:36 +0200 X-Google-Sender-Auth: Pu6NW8ophx34AI1VLfjsUvgl37o Message-ID: Subject: Re: 0.7 memory usage problem From: Peter Schuller To: user@cassandra.apache.org Content-Type: text/plain; charset=UTF-8 X-Virus-Checked: Checked by ClamAV on apache.org > Even I would like to add here something and correct me if I am wrong, I > downloaded 0.7 beta and ran it, just by chance I checked 'top' to see how > the new version is doing and there were 64 processes running though > Cassandra was on single node with default configuration options ( ran it as > is, as soon as I downloaded). No inserts done, no selects done nothing. I > don't think this is normal. I presume those are threads. It adds up; various stages have multiple threads in cassandra, and the JVM itself has a number of threads (e.g. GC threads, compiler threads). A 'jstack' on a freshly started trunk cassandra for me, grepping for 'prio', yields 92 threads the following. "Attach Listener" daemon prio=9 tid=0x0000000805476800 nid=0x80554e3c0 waiting on condition [0x0000000000000000] "Timer-1" prio=5 tid=0x00000009103ed800 nid=0x910777280 in Object.wait() [0x00007ffffa1a4000] "LB-TARGET:1" prio=5 tid=0x00000009103ee800 nid=0x910777b40 waiting on condition [0x00007ffffa2a5000] "LB-OPERATIONS:1" prio=5 tid=0x00000009103f0000 nid=0x910778400 waiting on condition [0x00007ffffa3a6000] "ACCEPT-localhost/127.0.0.1" prio=5 tid=0x00000009103f0800 nid=0x910778cc0 runnable [0x00007ffffa4a7000] "Timer-0" prio=5 tid=0x00000009103f1800 nid=0x9103d4ac0 in Object.wait() [0x00007ffffa5a8000] "GC inspection" prio=5 tid=0x00000009103f2000 nid=0x91051c540 in Object.wait() [0x00007ffffa6a9000] "CompactionExecutor:1" prio=1 tid=0x0000000911805800 nid=0x9111fb3c0 waiting on condition [0x00007ffffa7aa000] "PERIODIC-COMMIT-LOG-SYNCER" prio=5 tid=0x00000009103f3000 nid=0x91051ce00 waiting on condition [0x00007ffffa8ab000] "COMMIT-LOG-WRITER" prio=5 tid=0x00000009103f3800 nid=0x91051d6c0 waiting on condition [0x00007ffffa9ac000] "MISC_STAGE:1" prio=5 tid=0x00000009103f4800 nid=0x91051df80 waiting on condition [0x00007ffffaaad000] "MIGRATION_STAGE:1" prio=5 tid=0x00000009103f5000 nid=0x91051e840 waiting on condition [0x00007ffffabae000] "AE_SERVICE_STAGE:1" prio=5 tid=0x00000009103f6000 nid=0x91051f100 waiting on condition [0x00007ffffacaf000] "GOSSIP_STAGE:1" prio=5 tid=0x00000009103f6800 nid=0x91051f9c0 waiting on condition [0x00007ffffadb0000] "STREAM_STAGE:1" prio=5 tid=0x00000009103f7800 nid=0x910520280 waiting on condition [0x00007ffffaeb1000] "RESPONSE_STAGE:4" prio=5 tid=0x00000009103f8000 nid=0x910520b40 waiting on condition [0x00007ffffafb2000] "RESPONSE_STAGE:3" prio=5 tid=0x00000009103f9000 nid=0x910521400 waiting on condition [0x00007ffffb0b3000] "RESPONSE_STAGE:2" prio=5 tid=0x00000009103f9800 nid=0x910521cc0 waiting on condition [0x00007ffffb1b4000] "RESPONSE_STAGE:1" prio=5 tid=0x00000009103fa800 nid=0x9103c8900 waiting on condition [0x00007ffffb2b5000] "READ_STAGE:8" prio=5 tid=0x0000000910505000 nid=0x9103ce380 waiting on condition [0x00007ffffb3b6000] "READ_STAGE:7" prio=5 tid=0x0000000910505800 nid=0x9103cec40 waiting on condition [0x00007ffffb4b7000] "READ_STAGE:6" prio=5 tid=0x0000000910506800 nid=0x9103cf500 waiting on condition [0x00007ffffb5b8000] "READ_STAGE:5" prio=5 tid=0x0000000910507000 nid=0x9103cfdc0 waiting on condition [0x00007ffffb6b9000] "READ_STAGE:4" prio=5 tid=0x0000000801cbf000 nid=0x9103d0680 waiting on condition [0x00007ffffb7ba000] "READ_STAGE:3" prio=5 tid=0x0000000801cbf800 nid=0x9103d0f40 waiting on condition [0x00007ffffb8bb000] "READ_STAGE:2" prio=5 tid=0x0000000801cc0800 nid=0x9103d1800 waiting on condition [0x00007ffffb9bc000] "READ_STAGE:1" prio=5 tid=0x0000000801cc1000 nid=0x9103d20c0 waiting on condition [0x00007ffffbabd000] "MUTATION_STAGE:32" prio=5 tid=0x0000000801cc2000 nid=0x9103d2980 waiting on condition [0x00007ffffbbbe000] "MUTATION_STAGE:31" prio=5 tid=0x0000000801cc2800 nid=0x9103d3240 waiting on condition [0x00007ffffbcbf000] "MUTATION_STAGE:30" prio=5 tid=0x0000000801cc3800 nid=0x9103d3b00 waiting on condition [0x00007ffffbdc0000] "MUTATION_STAGE:29" prio=5 tid=0x0000000801cc4000 nid=0x9103d43c0 waiting on condition [0x00007ffffbec1000] "MUTATION_STAGE:28" prio=5 tid=0x0000000801cc5000 nid=0x9103c21c0 waiting on condition [0x00007ffffbfc2000] "MUTATION_STAGE:27" prio=5 tid=0x0000000801cc5800 nid=0x9103c2a80 waiting on condition [0x00007ffffc0c3000] "MUTATION_STAGE:26" prio=5 tid=0x0000000801cc6800 nid=0x9103c3340 waiting on condition [0x00007ffffc1c4000] "MUTATION_STAGE:25" prio=5 tid=0x0000000801cc7000 nid=0x9103c3c00 waiting on condition [0x00007ffffc2c5000] "MUTATION_STAGE:24" prio=5 tid=0x0000000801cc8000 nid=0x9103c44c0 waiting on condition [0x00007ffffc3c6000] "MUTATION_STAGE:23" prio=5 tid=0x0000000801cc8800 nid=0x9103c4d80 waiting on condition [0x00007ffffc4c7000] "MUTATION_STAGE:22" prio=5 tid=0x0000000801cc9800 nid=0x9103c5640 waiting on condition [0x00007ffffc5c8000] "MUTATION_STAGE:21" prio=5 tid=0x0000000801ccb000 nid=0x9103c5f00 waiting on condition [0x00007ffffc6c9000] "MUTATION_STAGE:20" prio=5 tid=0x0000000801ccb800 nid=0x9103c67c0 waiting on condition [0x00007ffffc7ca000] "MUTATION_STAGE:19" prio=5 tid=0x0000000801ccc800 nid=0x9103c7080 waiting on condition [0x00007ffffc8cb000] "MUTATION_STAGE:18" prio=5 tid=0x0000000801ccd000 nid=0x9103c7940 waiting on condition [0x00007ffffc9cc000] "MUTATION_STAGE:17" prio=5 tid=0x0000000801cce000 nid=0x9103c8200 waiting on condition [0x00007ffffcacd000] "MUTATION_STAGE:16" prio=5 tid=0x0000000801cce800 nid=0x91037ae40 waiting on condition [0x00007ffffcbce000] "MUTATION_STAGE:15" prio=5 tid=0x0000000801ccf800 nid=0x9103b98c0 waiting on condition [0x00007ffffcccf000] "MUTATION_STAGE:14" prio=5 tid=0x0000000801cd0000 nid=0x9103ba180 waiting on condition [0x00007ffffcdd0000] "MUTATION_STAGE:13" prio=5 tid=0x0000000801cd1000 nid=0x9103baa40 waiting on condition [0x00007ffffced1000] "MUTATION_STAGE:12" prio=5 tid=0x0000000801cd1800 nid=0x9103bb300 waiting on condition [0x00007ffffcfd2000] "MUTATION_STAGE:11" prio=5 tid=0x0000000801cd2800 nid=0x9103bbbc0 waiting on condition [0x00007ffffd0d3000] "MUTATION_STAGE:10" prio=5 tid=0x0000000801cd3000 nid=0x9103bc480 waiting on condition [0x00007ffffd1d4000] "MUTATION_STAGE:9" prio=5 tid=0x0000000801cd4000 nid=0x9103bcd40 waiting on condition [0x00007ffffd2d5000] "MUTATION_STAGE:8" prio=5 tid=0x0000000801cd4800 nid=0x9103bd600 waiting on condition [0x00007ffffd3d6000] "MUTATION_STAGE:7" prio=5 tid=0x0000000801cd5800 nid=0x9103bdec0 waiting on condition [0x00007ffffd4d7000] "MUTATION_STAGE:6" prio=5 tid=0x0000000801cd7000 nid=0x9103be780 waiting on condition [0x00007ffffd5d8000] "MUTATION_STAGE:5" prio=5 tid=0x0000000801cd7800 nid=0x9103bf040 waiting on condition [0x00007ffffd6d9000] "MUTATION_STAGE:4" prio=5 tid=0x0000000801cd8800 nid=0x908d16c80 waiting on condition [0x00007ffffd7da000] "MUTATION_STAGE:3" prio=5 tid=0x0000000801cd9000 nid=0x910374700 waiting on condition [0x00007ffffd8db000] "MUTATION_STAGE:2" prio=5 tid=0x0000000801cda000 nid=0x910374fc0 waiting on condition [0x00007ffffd9dc000] "MUTATION_STAGE:1" prio=5 tid=0x0000000801cda800 nid=0x910375880 waiting on condition [0x00007ffffdadd000] "SSTABLE-CLEANUP-TIMER" prio=5 tid=0x0000000801cdb800 nid=0x910376140 in Object.wait() [0x00007ffffdbde000] "SSTABLE-DELETER" prio=5 tid=0x0000000801cdc000 nid=0x910376a00 in Object.wait() [0x00007ffffdcdf000] "DroppedMessagesLogger" prio=5 tid=0x0000000801cdd000 nid=0x9103772c0 in Object.wait() [0x00007ffffdde0000] "MESSAGE-STREAMING-POOL:1" prio=5 tid=0x0000000801cdd800 nid=0x910377b80 waiting on condition [0x00007ffffdee1000] "EXPIRING-MAP-TIMER-2" daemon prio=5 tid=0x0000000801cde800 nid=0x910378440 in Object.wait() [0x00007ffffdfe2000] "EXPIRING-MAP-TIMER-1" daemon prio=5 tid=0x0000000801cdf000 nid=0x910378d00 in Object.wait() [0x00007ffffe0e3000] "CONSISTENCY-MANAGER:4" prio=5 tid=0x0000000801ce0000 nid=0x9103795c0 waiting on condition [0x00007ffffe1e4000] "CONSISTENCY-MANAGER:3" prio=5 tid=0x0000000801ce0800 nid=0x910379e80 waiting on condition [0x00007ffffe2e5000] "CONSISTENCY-MANAGER:2" prio=5 tid=0x0000000801ce1800 nid=0x91037a740 waiting on condition [0x00007ffffe3e6000] "CONSISTENCY-MANAGER:1" prio=5 tid=0x0000000801ce3000 nid=0x908d10540 waiting on condition [0x00007ffffe4e7000] "MEMTABLE-POST-FLUSHER:1" prio=5 tid=0x0000000801ce3800 nid=0x908d10e00 waiting on condition [0x00007ffffe5e8000] "FLUSH-WRITER-POOL:1" prio=5 tid=0x0000000801ce4800 nid=0x908d116c0 waiting on condition [0x00007ffffe6e9000] "FLUSH-SORTER-POOL:1" prio=5 tid=0x0000000801ce5000 nid=0x908d11f80 waiting on condition [0x00007ffffe7ea000] "FLUSH-TIMER" prio=5 tid=0x0000000801ce6000 nid=0x908d12840 in Object.wait() [0x00007ffffe8eb000] "DynamicEndpointSnitch" prio=5 tid=0x0000000801ce6800 nid=0x908d13100 in Object.wait() [0x00007ffffe9ec000] "RMI TCP Accept-0" daemon prio=5 tid=0x0000000801ce7800 nid=0x908d140c0 runnable [0x00007ffffebee000] "RMI TCP Accept-8080" daemon prio=5 tid=0x0000000801ce8000 nid=0x908d14980 runnable [0x00007ffffecef000] "RMI TCP Accept-0" daemon prio=5 tid=0x0000000801ce9000 nid=0x908d15240 runnable [0x00007ffffedf0000] "Low Memory Detector" daemon prio=5 tid=0x0000000801ce9800 nid=0x908d15cc0 runnable [0x0000000000000000] "CompilerThread1" daemon prio=9 tid=0x0000000801cea800 nid=0x908d16580 waiting on condition [0x0000000000000000] "CompilerThread0" daemon prio=9 tid=0x0000000801ceb000 nid=0x801c3c380 waiting on condition [0x0000000000000000] "Signal Dispatcher" daemon prio=9 tid=0x0000000801cec000 nid=0x801c3cc40 runnable [0x0000000000000000] "Surrogate Locker Thread (CMS)" daemon prio=5 tid=0x0000000801cec800 nid=0x801c3d500 waiting on condition [0x0000000000000000] "Finalizer" daemon prio=8 tid=0x0000000801ced800 nid=0x801c3ddc0 in Object.wait() [0x00007fffff3f6000] "Reference Handler" daemon prio=10 tid=0x0000000801cef000 nid=0x801c3e680 in Object.wait() [0x00007fffff4f7000] "main" prio=5 tid=0x0000000801cef800 nid=0x800c0ae40 runnable [0x00007fffffbfe000] "VM Thread" prio=9 tid=0x0000000801d22000 nid=0x801c3ef40 runnable "Gang worker#0 (Parallel GC Threads)" prio=9 tid=0x0000000801d26000 nid=0x801c41cc0 runnable "Gang worker#1 (Parallel GC Threads)" prio=9 tid=0x0000000801d25000 nid=0x801c41400 runnable "Gang worker#2 (Parallel GC Threads)" prio=9 tid=0x0000000801d24800 nid=0x801c40b40 runnable "Gang worker#3 (Parallel GC Threads)" prio=9 tid=0x0000000801d24000 nid=0x801c40280 runnable "Concurrent Mark-Sweep GC Thread" prio=9 tid=0x0000000801d23800 nid=0x801c3f800 runnable "VM Periodic Task Thread" prio=10 tid=0x0000000801d20800 nid=0x908d139c0 waiting on condition -- / Peter Schuller