Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Sat, 27 Sep 2014 04:56:34 +0000 (UTC)
From: "graham sanderson (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12727216.1405385811000.137539.1411793794130@Atlassian.JIRA>
In-Reply-To: <JIRA.12727216.1405385811000@Atlassian.JIRA>
References: <JIRA.12727216.1405385811000@Atlassian.JIRA>
 <JIRA.12727216.1405385811889@arcas>
Subject: [jira] [Comment Edited] (CASSANDRA-7546)
 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates
 memory
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14150409#comment-14150409 ] 

graham sanderson edited comment on CASSANDRA-7546 at 9/27/14 4:55 AM:
----------------------------------------------------------------------

Busy week - I I just did native_objects runs. It actually really helps out here a lot too - seems like native allocation starts taking a hit with too much concurrency.

I was about to do the hinting graphs, but cassandra-stress seems to be pulling the server names from the server (so I can't start it with one node down) - or maybe I can, and I should just ignore the errors (I just tried giving it 4/5 nodes on the command line)

What would you like me to do for n= ... I do have the full raw output for all these runs

!graphs2_7546.png!


was (Author: graham sanderson):
Busy week - I did the native_objects graphs. It actually really helps out here too - seems like native allocation starts taking a hit with too much concurrency.

I was about to do the hinting graphs, but cassandra-stress seems to be pulling the server names from the server (so I can't start it with one node down) - or maybe I can, and I should just ignore the errors (I just tried giving it 4/5 nodes on the command line)

What would you like me to do for n= ... I do have the full raw output for all these runs

!graphs2_7546.png!


> AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
> -----------------------------------------------------------------------------
>
>                 Key: CASSANDRA-7546
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: graham sanderson
>            Assignee: graham sanderson
>             Fix For: 2.1.1
>
>         Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, 7546.20_4.txt, 7546.20_5.txt, 7546.20_6.txt, 7546.20_7.txt, 7546.20_7b.txt, 7546.20_alt.txt, 7546.20_async.txt, 7546.21_v1.txt, graph2_7546.png, graphs1.png, hint_spikes.png, suggestion1.txt, suggestion1_21.txt, young_gen_gc.png
>
>
> In order to preserve atomicity, this code attempts to read, clone/update, then CAS the state of the partition.
> Under heavy contention for updating a single partition this can cause some fairly staggering memory growth (the more cores on your machine the worst it gets).
> Whilst many usage patterns don't do highly concurrent updates to the same partition, hinting today, does, and in this case wild (order(s) of magnitude more than expected) memory allocation rates can be seen (especially when the updates being hinted are small updates to different partitions which can happen very fast on their own) - see CASSANDRA-7545
> It would be best to eliminate/reduce/limit the spinning memory allocation whilst not slowing down the very common un-contended case.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)