cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Corentin Chary (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-12915) SASI: Index intersection can be very inefficient
Date Thu, 24 Nov 2016 13:55:59 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15693342#comment-15693342
] 

Corentin Chary commented on CASSANDRA-12915:
--------------------------------------------

I also have no idea what the primary expression was supposed to be ... I guess we can drop
it.

It's true what we should do something before fetching the tokens on every expressions. There
are other things I was thinking about:
* Ordering expressions with a score (EQ first, then something like estimated result rows)
* Skipping the following indexes if we already found one with less tokens than command.limits().count()

Regarding 100000, that's the value at which I found iterating on the disk token index was
significantly slower than doing anything else. I guess using limits() could work too. Regarding
the factor, that's true, but I'm not sore what would be a better heuristic.

In my case I do not use only EQ relations, in some cases these may be prefixes (https://github.com/criteo/biggraphite/blob/9d436b6212b1f603fd3d4eff0b8c465d9a602454/biggraphite/drivers/cassandra.py,
https://github.com/criteo/biggraphite/blob/fcbb91bbad2149334d67082cc7f1c1573215c497/CASSANDRA_DESIGN.md)

So, what we need first is something like getEstimatedResultRows. Do you have a good suggestion
to do that without doing the search ? I didn't find much appart from the index file size.



> SASI: Index intersection can be very inefficient
> ------------------------------------------------
>
>                 Key: CASSANDRA-12915
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12915
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: sasi
>            Reporter: Corentin Chary
>             Fix For: 3.x
>
>
> It looks like RangeIntersectionIterator.java and be pretty inefficient in some cases.
Let's take the following query:
> SELECT data FROM table WHERE index1 = 'foo' AND index2 = 'bar';
> In this case:
> * index1 = 'foo' will match 2 items
> * index2 = 'bar' will match ~300k items
> On my setup, the query will take ~1 sec, most of the time being spent in disk.TokenTree.getTokenAt().
> if I patch RangeIntersectionIterator so that it doesn't try to do the intersection (and
effectively only use 'index1') the query will run in a few tenth of milliseconds.
> I see multiple solutions for that:
> * Add a static thresold to avoid the use of the index for the intersection when we know
it will be slow. Probably when the range size factor is very small and the range size is big.
> * CASSANDRA-10765



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message