Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Wed, 23 Jul 2014 22:26:38 +0000 (UTC)
From: "graham sanderson (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12727216.1405385811889.33169.1406154398911@arcas>
In-Reply-To: <JIRA.12727216.1405385811889@arcas>
References: <JIRA.12727216.1405385811889@arcas>
Subject: [jira] [Commented] (CASSANDRA-7546)
 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates
 memory
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14072465#comment-14072465 ] 

graham sanderson commented on CASSANDRA-7546:
---------------------------------------------

OK - I had something else come up today, but yeah I realized my math was wrong... it is certainly a bit of a massage the correct fidelity of information within 32 bits, without overflowing too soon, or not having enough padding so that bursty allocation under the sustained limit causes problems.

bq. It is monotonic; that's its main purpose.

I guess we (me?) are being pedantic here... it is the low 64 bits of a monotonic number - (even this was broken on early OS/JVM combinations due to bugs, however we can take that as fact now I think); what the actual number is is undefined. It does seem on UNIX variants appear to be rebased to nanonseconds since epoch, and probably on all modern systems is some counter that was reset at least on power cycle, so you are probably ok. In any case, doing the right thing is pretty much always trivial (assuming you don't expect your JVM to run for 200+ years)

--

As an aside, can you give me a hint as to how long you expect AtomicSorted/BTreeColumns to last... tuning does seem critical here, since wasting 100M would probably be a reasonable value, but I don't know in practice if something else would likely end up flushing the memtable before it ever got that far.

> AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
> -----------------------------------------------------------------------------
>
>                 Key: CASSANDRA-7546
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: graham sanderson
>            Assignee: graham sanderson
>         Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, 7546.20_alt.txt, suggestion1.txt, suggestion1_21.txt
>
>
> In order to preserve atomicity, this code attempts to read, clone/update, then CAS the state of the partition.
> Under heavy contention for updating a single partition this can cause some fairly staggering memory growth (the more cores on your machine the worst it gets).
> Whilst many usage patterns don't do highly concurrent updates to the same partition, hinting today, does, and in this case wild (order(s) of magnitude more than expected) memory allocation rates can be seen (especially when the updates being hinted are small updates to different partitions which can happen very fast on their own) - see CASSANDRA-7545
> It would be best to eliminate/reduce/limit the spinning memory allocation whilst not slowing down the very common un-contended case.


--
This message was sent by Atlassian JIRA
(v6.2#6252)