cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jay Zhuang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-12453) AutoSavingCache does not store required keys making RowCacheTests Flaky
Date Fri, 23 Dec 2016 01:33:58 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15771572#comment-15771572
] 

Jay Zhuang commented on CASSANDRA-12453:
----------------------------------------

The issue can be consistently reproduced on machine with > 16 CPUs (which will use [64
cache segments|https://github.com/snazy/ohc/blob/develop/ohc-core/src/main/java/org/caffinitas/ohc/OHCacheBuilder.java#L159]),
or override the [{{segmentCount}}|https://github.com/snazy/ohc/blob/develop/ohc-core/src/main/java/org/caffinitas/ohc/OHCacheBuilder.java#L162]
to [64 | https://github.com/cooldoger/cassandra/commit/53facbb6e75a9adf69be8622ea0d951d3996cc8a]:
{code}
    [junit] Testcase: testRowCachePartialLoad(org.apache.cassandra.db.RowCacheTest):    FAILED
    [junit] expected:<50> but was:<48>
    [junit] junit.framework.AssertionFailedError: expected:<50> but was:<48>
    [junit]     at org.apache.cassandra.db.RowCacheTest.rowCacheLoad(RowCacheTest.java:413)
    [junit]     at org.apache.cassandra.db.RowCacheTest.testRowCachePartialLoad(RowCacheTest.java:231)
{code}

When there're only 100 keys are [hashed to 64 segments|https://github.com/snazy/ohc/blob/develop/ohc-core/src/main/java/org/caffinitas/ohc/linked/OHCacheLinkedImpl.java#L275],
16 segments are actually empty. The hotKeyIterator try to get [one key per segments|https://github.com/snazy/ohc/blob/develop/ohc-core/src/main/java/org/caffinitas/ohc/linked/OHCacheLinkedImpl.java#L1396],
which will only get 48.

The fix is to increase the data size to cover more segments, so the hotKeyIterator is able
to get 50 keys. I don't think it will impact Cassandra, as in production, the cache segments
should be filled with more data, even get less cache keys should not be a problem.
The fix is attached, please review.

> AutoSavingCache does not store required keys making RowCacheTests Flaky
> -----------------------------------------------------------------------
>
>                 Key: CASSANDRA-12453
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12453
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: sankalp kohli
>            Assignee: Jay Zhuang
>            Priority: Minor
>             Fix For: 2.2.x, 3.0.x
>
>         Attachments: 12453-2.2.txt
>
>
> RowCacheTests were flaky and while investigating, I found that it does not store all
the keys to disk. 
> The reason is that we use  OHCache and call hotKeyIterator on it. This is not guaranteed
to return the number of keys we want. Here is the documentation from OHCache 
> /**
>      * Builds an iterator over the N most recently used keys returning deserialized objects.
>      * You must call {@code close()} on the returned iterator.
>      * <p>
>      *     Note: During a rehash, the implementation might return keys twice or not at
all.
>      * </p>
>      */
>     CloseableIterator<K> hotKeyIterator(int n);



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message