cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
Date Wed, 23 Jul 2014 23:26:39 GMT


Benedict commented on CASSANDRA-7546:

Yes, I'm referring to the memtables - an AtomicSortedColumns instance lives until its containing
memtable is flushed. 100MB/s is around 1M snaptree node allocations, so that is maybe a little
high for deciding there's too much competition (although with ~ 1000 items present this is
only 100k inserts), so how about we fix it to 10MB/s, to be exceeded by 10Mb. We could certainly
hit 100MB of waste, no trouble (under high competition we'll see orders of magnitude more
wasted than used, and memtables usually store 1Gb+), but I think it's better to trigger a
little more readily

> AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
> -----------------------------------------------------------------------------
>                 Key: CASSANDRA-7546
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: graham sanderson
>            Assignee: graham sanderson
>         Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, 7546.20_alt.txt, suggestion1.txt,
> In order to preserve atomicity, this code attempts to read, clone/update, then CAS the
state of the partition.
> Under heavy contention for updating a single partition this can cause some fairly staggering
memory growth (the more cores on your machine the worst it gets).
> Whilst many usage patterns don't do highly concurrent updates to the same partition,
hinting today, does, and in this case wild (order(s) of magnitude more than expected) memory
allocation rates can be seen (especially when the updates being hinted are small updates to
different partitions which can happen very fast on their own) - see CASSANDRA-7545
> It would be best to eliminate/reduce/limit the spinning memory allocation whilst not
slowing down the very common un-contended case.

This message was sent by Atlassian JIRA

View raw message