cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "mck (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-14247) SASI tokenizer for simple delimiter based entries
Date Thu, 15 Mar 2018 03:57:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399884#comment-16399884
] 

mck edited comment on CASSANDRA-14247 at 3/15/18 3:56 AM:
----------------------------------------------------------

[~mkjellman],
{quote}one thing that stuck out to me was this while loop that didn't actually "do" anything
but it does do something... could you at least throw a comment in just to make it a bit more
readable?{quote}

comment thrown in :-)

The byte buffer approach is pushed to the trunk_14247 and cassandra-3.11_14247 branches. 

The rationale to adding this patch also to cassandra-3.11 is it's an important stability workaround
to {{\{mode:CONTAINS\}}}, and is a standalone class, annotated as {{@Beta}}, that does not
touch any other code .

The following patches have been submitted:

|| branch || testall || dtest ||
| [cassandra-3.11_14247|https://github.com/thelastpickle/cassandra/tree/mck/cassandra-3.11_14247]
| [testall|https://circleci.com/gh/thelastpickle/cassandra/tree/mck%2Fcassandra-3.11_14247]
| [dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/512]
|
| [trunk_14247|https://github.com/thelastpickle/cassandra/tree/mck/trunk_14247]	| [testall|https://circleci.com/gh/thelastpickle/cassandra/tree/mck%2Ftrunk_14247]
| [dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/513]
|



was (Author: michaelsembwever):
[~mkjellman],
{quote}one thing that stuck out to me was this while loop that didn't actually "do" anything
but it does do something... could you at least throw a comment in just to make it a bit more
readable?{quote}

comment thrown in :-)

The byte buffer approach is pushed to the trunk_14247 and cassandra-3.11_14247 branches. 

The rationale to adding this patch also to cassandra-3.11 is it's an important stability workaround
to {{mode: CONTAINS}}, and it is an additional standalone class that does not touch other
code which has been annotated as {{@Beta}}.

The following patches have been submitted:

|| branch || testall || dtest ||
| [cassandra-3.11_14247|https://github.com/thelastpickle/cassandra/tree/mck/cassandra-3.11_14247]
| [testall|https://circleci.com/gh/thelastpickle/cassandra/tree/mck%2Fcassandra-3.11_14247]
| [dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/512]
|
| [trunk_14247|https://github.com/thelastpickle/cassandra/tree/mck/trunk_14247]	| [testall|https://circleci.com/gh/thelastpickle/cassandra/tree/mck%2Ftrunk_14247]
| [dtest|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/513]
|


> SASI tokenizer for simple delimiter based entries
> -------------------------------------------------
>
>                 Key: CASSANDRA-14247
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-14247
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: sasi
>            Reporter: mck
>            Assignee: mck
>            Priority: Major
>             Fix For: 4.0, 3.11.x
>
>
> Currently SASI offers only two tokenizer options:
>  - NonTokenizerAnalyser
>  - StandardAnalyzer
> The latter is built upon Snowball, powerful for human languages but overkill for simple
tokenization.
> A simple tokenizer is proposed here. The need for this arose as a workaround of CASSANDRA-11182,
and to avoid the disk usage explosion when having to resort to {{CONTAINS}}. See https://github.com/openzipkin/zipkin/issues/1861
> Example use of this would be:
> {code}
> CREATE CUSTOM INDEX span_annotation_query_idx 
>     ON zipkin2.span (annotation_query) USING 'org.apache.cassandra.index.sasi.SASIIndex'

>     WITH OPTIONS = {
>         'analyzer_class': 'org.apache.cassandra.index.sasi.analyzer.DelimiterAnalyzer',

>         'delimiter': '░',
>         'case_sensitive': 'true', 
>         'mode': 'prefix', 
>         'analyzed': 'true'};
> {code}
> Original credit for this work goes to https://github.com/zuochangan



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message