Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BA3C017F0D for ; Fri, 27 Feb 2015 06:07:27 +0000 (UTC) Received: (qmail 41616 invoked by uid 500); 27 Feb 2015 06:07:05 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 41574 invoked by uid 500); 27 Feb 2015 06:07:05 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 41548 invoked by uid 99); 27 Feb 2015 06:07:05 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 27 Feb 2015 06:07:05 +0000 Date: Fri, 27 Feb 2015 06:07:05 +0000 (UTC) From: "Phil Yang (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Issue Comment Deleted] (CASSANDRA-8860) Too many java.util.HashMap$Entry objects in heap MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-8860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phil Yang updated CASSANDRA-8860: --------------------------------- Comment: was deleted (was: The heapdump is still being uploaded, I find these Entry objects is indeed generated by SizeTieredCompactionStrategy.createOverlapChain. So do you still need the heapdump? Besides, in my cluster there are only two table with SizeTieredCompactionStrategy, one is System.paxos which has no data, the other has about 1100 tables in total. {noformat} CREATE TABLE dict_voice.voice ( key bigint PRIMARY KEY, data blob ) WITH COMPACT STORAGE AND bloom_filter_fp_chance = 0.1 AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}' AND comment = '' AND compaction = {'min_threshold': '4', 'class': 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy', 'max_threshold': '32'} AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'} AND dclocal_read_repair_chance = 0.0 AND default_time_to_live = 0 AND gc_grace_seconds = 864000 AND max_index_interval = 2048 AND memtable_flush_period_in_ms = 0 AND min_index_interval = 128 AND read_repair_chance = 0.0 AND speculative_retry = '99.0PERCENTILE'; {noformat} SSTables in each level: [1, 10, 103/100, 904, 128, 0, 0, 0, 0]) > Too many java.util.HashMap$Entry objects in heap > ------------------------------------------------ > > Key: CASSANDRA-8860 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8860 > Project: Cassandra > Issue Type: Bug > Environment: Cassandra 2.1.3, jdk 1.7u51 > Reporter: Phil Yang > Assignee: Marcus Eriksson > Fix For: 2.1.4 > > Attachments: cassandra-env.sh, cassandra.yaml, jmap.txt, jstack.txt > > > While I upgrading my cluster to 2.1.3, I find some nodes (not all) may have GC issue after the node restarting successfully. Old gen grows very fast and most of the space can not be recycled after setting its status to normal immediately. The qps of both reading and writing are very low and there is no heavy compaction. > Jmap result seems strange that there are too many java.util.HashMap$Entry objects in heap, where in my experience the "[B" is usually the No1. > If I downgrade it to 2.1.1, this issue will not appear. > I uploaded conf files and jstack/jmap outputs. I'll upload heap dump if someone need it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)