cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phil Yang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-8860) Too many java.util.HashMap$Entry objects in heap
Date Sun, 01 Mar 2015 07:43:05 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-8860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14342032#comment-14342032
] 

Phil Yang edited comment on CASSANDRA-8860 at 3/1/15 7:42 AM:
--------------------------------------------------------------

Furthermore, if my algorithm is correct, my patch-v2 still need a 2-d loop to get the undirected
graph. The space complexity is still O(N^2) if all the SSTables overlap each other and the
graph is a complete graph. So I think there is a way to optimize it:

Use a 2-d bitmap matrix with the boolean which means whether coldSSTables[i] overlaps coldSSTables[j]
to replace the 2-d collection of SSTableReader. Each HashMap$Entry need 32*8=256 bits but
each bitmap boolean needs only 1 bit. This method may need more memory in some case, but need
much less in the worst case.

If my algorithm is not wrong, we can discuss if this optimization is needed or not :)


was (Author: yangzhe1991):
Furthermore, if my algorithm is correct, my patch-v2 still need a 2-d loop to get the undirected
graph. The space complexity is still O(N^2) if all the SSTables overlap each other. So I think
there is a way to optimize it:

Use a 2-d bitmap matrix with the boolean which means whether coldSSTables[i] overlaps coldSSTables[j]
to replace the 2-d collection of SSTableReader. Each HashMap$Entry need 32*8=256 bits but
each bitmap boolean needs only 1 bit. This method may need more memory in some case, but need
much less in the worst case.

If my algorithm is not wrong, we can discuss if this optimization is needed or not :)

> Too many java.util.HashMap$Entry objects in heap
> ------------------------------------------------
>
>                 Key: CASSANDRA-8860
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8860
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: Cassandra 2.1.3, jdk 1.7u51
>            Reporter: Phil Yang
>            Assignee: Phil Yang
>             Fix For: 2.1.4
>
>         Attachments: 8860-v2.txt, 8860.txt, cassandra-env.sh, cassandra.yaml, jmap.txt,
jstack.txt, jstat-afterv1.txt, jstat-afterv2.txt, jstat-before.txt
>
>
> While I upgrading my cluster to 2.1.3, I find some nodes (not all) may have GC issue
after the node restarting successfully. Old gen grows very fast and most of the space can
not be recycled after setting its status to normal immediately. The qps of both reading and
writing are very low and there is no heavy compaction.
> Jmap result seems strange that there are too many java.util.HashMap$Entry objects in
heap, where in my experience the "[B" is usually the No1.
> If I downgrade it to 2.1.1, this issue will not appear.
> I uploaded conf files and jstack/jmap outputs. I'll upload heap dump if someone need
it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message