cassandra-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nimi Wariboko Jr <n...@channelmeter.com>
Subject Re: BufferPool always allocates on heap?
Date Wed, 22 Jun 2016 15:44:54 GMT
Hi Stefania,

Thanks for the explanation - I think it may be more likely I'm hitting
MEMORY_USAGE_THRESHOLD
(file_cache_size_in_mb, which is 512mb by default), as the table itself is
very large (a couple TB over 12 nodes). In that case, due to the changes
outlined CASSANDRA-9548/CASSANDRA-8709/CASSANDRA-5863, there should
probably be a warning in 3.6 that buffer_pool_use_heap_if_exhausted should
be set to false if you still have some tables on SnappyCompression.

Nimi

On Wed, Jun 22, 2016 at 12:14 AM, Stefania Alborghetti <
stefania.alborghetti@datastax.com> wrote:

> Hi Nimi
>
> I am not familiar with how CASSANDRA-5863 uses the buffer pool yet, but I
> do know how the buffer pool works
>
> It has a local pool, which is thread local, and takes chunks from a global
> pool. These are allocated off heap and aligned to the page cache
> boundaries, see here
> <
> https://github.com/apache/cassandra/blob/30bb255ec9fb36ace2aab51474bd3bfb9bbd3bed/src/java/org/apache/cassandra/utils/memory/BufferPool.java#L276
> >.
>
>
> The buffer pool will allocate according to
> buffer_pool_use_heap_if_exhausted
> only in two cases:
>
>    - The global pool has run out of chunks, which happens when there is an
>    OOM exception or when MEMORY_USAGE_THRESHOLD is exhausted, see here
>    <
> https://github.com/apache/cassandra/blob/30bb255ec9fb36ace2aab51474bd3bfb9bbd3bed/src/java/org/apache/cassandra/utils/memory/BufferPool.java#L259
> >
>    .
>    - The requested buffer size is larger than a chunk, currently
>    CHUNK_SIZE, 64 kb.
>
> To understand this behaviour look at maybeTakeFromPool
> <
> https://github.com/apache/cassandra/blob/30bb255ec9fb36ace2aab51474bd3bfb9bbd3bed/src/java/org/apache/cassandra/utils/memory/BufferPool.java#L124
> >(),
> the line that takes from the local pool is the last one and follow from
> there.
>
> The original ticket for the buffer pool is CASSANDRA-8897
> <https://issues.apache.org/jira/browse/CASSANDRA-8897>if you need more
> details, and there is CASSANDRA-9468
> <https://issues.apache.org/jira/browse/CASSANDRA-9468> to extend it to
> allow chunks with bigger or smaller sizes (point 4).
>
> I've left a comment on CASSANDRA-11993.
>
> I hope this helps,
> Stefania
>
>
> On Wed, Jun 22, 2016 at 4:54 AM, Nimi Wariboko Jr <nimi@channelmeter.com>
> wrote:
>
> > Hi,
> >
> > I'm trying to debug an issue I'm having with Cassandra 3.6+ and Snappy
> > compressed tables - CASSANDRA-11993 (that may be related to
> > CASSANDRA-5863).
> >
> > I'm probably 3% familiar with Cassandra internals, so apologies if I'm
> > wrong here - I was looking at a change in CASSANDRA-5863 where a cache
> was
> > added to the SSTableIterator code path, which may have
> > included org.apache.cassandra.utils.memory.BufferPool into the read path.
> >
> > My confusion comes from here - AFAICT, when the BufferPool needs to
> > allocate more memory, it calls allocate(int size, boolean onHeap) (
> >
> >
> https://github.com/apache/cassandra/blob/30bb255ec9fb36ace2aab51474bd3bfb9bbd3bed/src/java/org/apache/cassandra/utils/memory/BufferPool.java#L82
> > ).
> > The value for whether onHeap is true is
> > constant ALLOCATE_ON_HEAP_WHEN_EXAHUSTED.
> >
> > ALLOCATE_ON_HEAP_WHEN_EXAHUSTED is defined by the yaml
> > setting buffer_pool_use_heap_if_exhausted which is true by default (and
> was
> > so in my install).
> >
> > Further still (
> >
> >
> https://github.com/apache/cassandra/blob/30bb255ec9fb36ace2aab51474bd3bfb9bbd3bed/src/java/org/apache/cassandra/utils/memory/BufferPool.java
> > ),
> > I can't see any logic where BufferPool will actually allocate off heap
> > memory - it seems, due to the buffer_pool_use_heap_if_exhausted setting,
> > always allocate memory on heap. It never checks if the off heap memory is
> > exhausted (AFAICT from these 3 files).
> >
> > Sure enough, if I set buffer_pool_use_heap_if_exhausted, my issue at
> > CASSANDRA-11993 goes away (I think? I'm having a number of issues lately
> > and I may have traded one exception for another).
> >
> > Is this the correct behavior of buffer_pool_use_heap_if_exhausted?
> >
> > Nimi
> >
>
>
>
> --
>
>
> [image: datastax_logo.png] <http://www.datastax.com/>
>
> Stefania Alborghetti
>
> Apache Cassandra Software Engineer
>
> |+852 6114 9265| stefania.alborghetti@datastax.com
>
>
> [image: cassandrasummit.org/Email_Signature]
> <http://cassandrasummit.org/Email_Signature>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message