cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Corentin Chary (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-12915) SASI: Index intersection can be very inefficient
Date Sun, 11 Dec 2016 16:35:58 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15739992#comment-15739992
] 

Corentin Chary commented on CASSANDRA-12915:
--------------------------------------------

I think I found something closer to what I originally noticed in my real data set. 

{code}
import sys
import random

def main(args):
    n = int(args[1])

    for i in xrange(n):
        a = '0'
        b = '%s.%s' % (i % 10, i)
        c = i % (n / 10) + random.randint(0, 10)
        print ("%s,%s,%s,%s,%s" % (i, a, b, c, i))

if __name__ == '__main__':
    main(sys.argv)
{code}

{code}
PAGING OFF;
SELECT * FROM test.test WHERE b LIKE '5%' AND c LIKE '4485%' LIMIT 1000 ALLOW FILTERING;
{code}

Two main differences:
- c LIKE '4485%'  will return more token (100+)
- using LIKE instead of EQ

Without the patch is seems to take 400-200ms, with it takes around 100ms (I doing that with
cqlsh, would you have something ready to use to better check the latency of a single query
?). visualvm seems to point at performSkipTo(). My theory is that LookupIntersectionIterator
the inner range.skipTo isn't as effecient as we think it is when range contains millions of
items.

> SASI: Index intersection can be very inefficient
> ------------------------------------------------
>
>                 Key: CASSANDRA-12915
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12915
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: sasi
>            Reporter: Corentin Chary
>             Fix For: 3.x
>
>
> It looks like RangeIntersectionIterator.java and be pretty inefficient in some cases.
Let's take the following query:
> SELECT data FROM table WHERE index1 = 'foo' AND index2 = 'bar';
> In this case:
> * index1 = 'foo' will match 2 items
> * index2 = 'bar' will match ~300k items
> On my setup, the query will take ~1 sec, most of the time being spent in disk.TokenTree.getTokenAt().
> if I patch RangeIntersectionIterator so that it doesn't try to do the intersection (and
effectively only use 'index1') the query will run in a few tenth of milliseconds.
> I see multiple solutions for that:
> * Add a static thresold to avoid the use of the index for the intersection when we know
it will be slow. Probably when the range size factor is very small and the range size is big.
> * CASSANDRA-10765



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message