cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Stupp (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
Date Thu, 04 Dec 2014 22:21:14 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14234696#comment-14234696
] 

Robert Stupp edited comment on CASSANDRA-7438 at 12/4/14 10:20 PM:
-------------------------------------------------------------------

Just pushed some OHC additions to github:
* key-iterator (used by CacheService class to invalidate column families)
* (de)serialization of cache content to disk using direct I/O from off-heap.  Means that the
row cache content does not need to go though the heap for serialization and deserialization.
Compression should also be possible in off-heap using the static methods in Snappy class since
these expect direct buffers so there's nearly no pressure for that on the heap. Background:
the implementation basically "lies" the address and length of the cache entry into DirectByteBuffer
class so FileChannel is able to read into it/write from it.

edit: s/hash/cache/


was (Author: snazy):
Just pushed some OHC additions to github:
* key-iterator (used by CacheService class to invalidate column families)
* (de)serialization of cache content to disk using direct I/O from off-heap.  Means that the
row cache content does not need to go though the heap for serialization and deserialization.
Compression should also be possible in off-heap using the static methods in Snappy class since
these expect direct buffers so there's nearly no pressure for that on the heap. Background:
the implementation basically "lies" the address and length of the hash entry into DirectByteBuffer
class so FileChannel is able to read into it/write from it.


> Serializing Row cache alternative (Fully off heap)
> --------------------------------------------------
>
>                 Key: CASSANDRA-7438
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7438
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>         Environment: Linux
>            Reporter: Vijay
>            Assignee: Vijay
>              Labels: performance
>             Fix For: 3.0
>
>         Attachments: 0001-CASSANDRA-7438.patch, tests.zip
>
>
> Currently SerializingCache is partially off heap, keys are still stored in JVM heap as
BB, 
> * There is a higher GC costs for a reasonably big cache.
> * Some users have used the row cache efficiently in production for better results, but
this requires careful tunning.
> * Overhead in Memory for the cache entries are relatively high.
> So the proposal for this ticket is to move the LRU cache logic completely off heap and
use JNI to interact with cache. We might want to ensure that the new implementation match
the existing API's (ICache), and the implementation needs to have safe memory access, low
overhead in memory and less memcpy's (As much as possible).
> We might also want to make this cache configurable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message