cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Stupp (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
Date Mon, 24 Nov 2014 08:59:13 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14222799#comment-14222799
] 

Robert Stupp edited comment on CASSANDRA-7438 at 11/24/14 8:58 AM:
-------------------------------------------------------------------

rehashing: growing (x2) is already implemented, shrinking (/2) shouldn't be a big issue, too.
The implementation only locks the currently processed partitions during rehash.
"put" operation: fixed (was definitely a bug), cleanup is running concurrently and trigger
on "out of memory" condition
block sizes: will give it a try (fixed vs. different sizes vs. variable sized (no blocks))
per-partition locks: already thought about it - not sure whether it's worth the additional
RW-lock overhead since partition lock time is very low during normal operation
metrics: some (very basic) metrics are already in it - will add some more timer metrics (configurable)

[~vijay2win@yahoo.com] can you catch {{OutOfMemoryError}} for Unsafe.allocate() ? It should
not go up the whole call stack as is to prevent C* handling that as "Java heap full".


was (Author: snazy):
rehashing: growing (x2) is already implemented, shrinking (/2) shouldn't be a big issue, too.
The implementation only locks the currently processed partitions during rehash.
"put" operation: fixed (was definitely a bug), cleanup is running concurrently and trigger
on "out of memory" condition
block sizes: will give it a try (fixed vs. different sizes vs. variable sized (no blocks))
per-partition locks: already thought about it - not sure whether it's worth the additional
RW-lock overhead since partition lock time is very low during normal operation
metrics: some (very basic) metrics are already in it - will add some more timer metrics (configurable)

[~vijay2win@yahoo.com] can you catch {{OutOfMemoryError}} for Unsafe.allocate() ? It should
not go up the whole call stack.

> Serializing Row cache alternative (Fully off heap)
> --------------------------------------------------
>
>                 Key: CASSANDRA-7438
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>         Environment: Linux
>            Reporter: Vijay
>            Assignee: Vijay
>              Labels: performance
>             Fix For: 3.0
>
>         Attachments: 0001-CASSANDRA-7438.patch
>
>
> Currently SerializingCache is partially off heap, keys are still stored in JVM heap as
BB, 
> * There is a higher GC costs for a reasonably big cache.
> * Some users have used the row cache efficiently in production for better results, but
this requires careful tunning.
> * Overhead in Memory for the cache entries are relatively high.
> So the proposal for this ticket is to move the LRU cache logic completely off heap and
use JNI to interact with cache. We might want to ensure that the new implementation match
the existing API's (ICache), and the implementation needs to have safe memory access, low
overhead in memory and less memcpy's (As much as possible).
> We might also want to make this cache configurable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message