cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Voytek Jarnot (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-12877) SASI index throwing AssertionError on index creation
Date Fri, 04 Nov 2016 21:12:58 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637691#comment-15637691
] 

Voytek Jarnot edited comment on CASSANDRA-12877 at 11/4/16 9:12 PM:
--------------------------------------------------------------------

Attached full log output.  Fresh build of cassandra-3.X; fresh install, fresh keyspace (SimpleStrategy,
RF 1).

1) built/installed 3.10-SNAPSHOT from git branch cassandra-3.X
2) created keyspace (SimpleStrategy, RF 1)
3) created table: (simplified version below, many more valX columns present)
{quote}
CREATE TABLE mytable (
    id1 text,
    id2 text,
    id3 date,
    id4 timestamp,
    id5 text,
    val1 text,
    val2 text,
    val3 text,
    task_id text,
    document_nbr text,
    val5 text,
    PRIMARY KEY ((id1, id2), id3, id4, id5)
) WITH CLUSTERING ORDER BY (id3 DESC, id4 DESC, id5 ASC)
{quote}

4) created materialized view:
{quote}
CREATE MATERIALIZED VIEW mytable_by_task_id AS
    SELECT *
    FROM mytable
    WHERE id1 IS NOT NULL AND id2 IS NOT NULL AND id3 IS NOT NULL AND id4 IS NOT NULL AND
id5 IS NOT NULL AND task_id IS NOT NULL
    PRIMARY KEY (task_id, id3, id4, id1, id2, id5)
    WITH CLUSTERING ORDER BY (id3 DESC, id4 DESC, id1 ASC, id2 ASC, id5 ASC)
{quote}
5) inserted 27 million "rows" (i.e., unique values for id5)
6) create index attempt
{quote}
create custom index idx_ar_document_nbr on test_table(document_nbr) using 'org.apache.cassandra.index.sasi.SASIIndex';
{quote}
7) no error in cqlsh, logged errors attached.

Beginning to suspect CASSANDRA-11990 ... but don't have enough internals-knowledge to do much
more than guess.


was (Author: voytek.jarnot):
Attached full log output.  Fresh build of cassandra-3.X; fresh install, fresh keyspace (SimpleStrategy,
RF 1).

1) built/installed 3.10-SNAPSHOT from git branch cassandra-3.X
2) created keyspace (SimpleStrategy, RF 1)
3) created table: (simplified version below, many more valX columns present)
CREATE TABLE mytable (
    id1 text,
    id2 text,
    id3 date,
    id4 timestamp,
    id5 text,
    val1 text,
    val2 text,
    val3 text,
    task_id text,
    document_nbr text,
    val5 text,
    PRIMARY KEY ((id1, id2), id3, id4, id5)
) WITH CLUSTERING ORDER BY (id3 DESC, id4 DESC, id5 ASC)
4) created materialized view:
CREATE MATERIALIZED VIEW mytable_by_task_id AS
    SELECT *
    FROM mytable
    WHERE id1 IS NOT NULL AND id2 IS NOT NULL AND id3 IS NOT NULL AND id4 IS NOT NULL AND
id5 IS NOT NULL AND task_id IS NOT NULL
    PRIMARY KEY (task_id, id3, id4, id1, id2, id5)
    WITH CLUSTERING ORDER BY (id3 DESC, id4 DESC, id1 ASC, id2 ASC, id5 ASC)
5) inserted 27 million "rows" (i.e., unique values for id5)
6) create index attempt
create custom index idx_ar_document_nbr on test_table(document_nbr) using 'org.apache.cassandra.index.sasi.SASIIndex';

7) no error in cqlsh, logged errors attached.

Beginning to suspect CASSANDRA-11990 ... but don't have enough internals-knowledge to do much
more than guess.

> SASI index throwing AssertionError on index creation
> ----------------------------------------------------
>
>                 Key: CASSANDRA-12877
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12877
>             Project: Cassandra
>          Issue Type: Bug
>          Components: sasi
>         Environment: 3.9 and 3.10 tested on both linux and osx
>            Reporter: Voytek Jarnot
>         Attachments: idx-stacktrace-03-nov-2016.txt, idx-stacktrace-04-nov-2016.txt
>
>
> Possibly a 3.10 regression?
> I built and installed a 3.10 snapshot (built 03-Nov-2016) to get around CASSANDRA-11670,
CASSANDRA-12689, and CASSANDRA-12223 which are holding me back when using 3.9. Edit to add:
3 node cluster, replication factor of 2.
> Would like to state up front that I can't duplicate this with a lightweight throwaway
test, which is frustrating, but it keeps hitting me on our dev cluster.  It may require a
certain amount of data present (or perhaps a high number of nulls in the indexed column) -
never had any luck duplicating with the table shown below.
> Table roughly resembles the following, with many more 'valx' columns:
> CREATE TABLE idx_test_table (
>     id1 text,
>     id2 text,
>     id3 text,
>     id4 text,
>     val1 text,
>     val2 text,
>     PRIMARY KEY ((id1, id2), id3, id4)
> ) WITH CLUSTERING ORDER BY (id3 DESC, id4 ASC);
> CREATE CUSTOM INDEX idx_test_index ON idx_test_table (val2) USING 'org.apache.cassandra.index.sasi.SASIIndex';
> The error below occurs in 3.10, but not in 3.9; it occurs whether I insert a bunch of
dev data and then create the index, or whether I create the index and then insert a bunch
of test data.
> {quote}
> INFO  [MemtableFlushWriter:5] 2016-11-03 21:00:19,416 PerSSTableIndexWriter.java:284
- Scheduling index flush to /u01/cassandra-data/data/essatc1/audit_record-520c1dc0a1e411e691db0bd4b103bd15/mc-266-big-SI_idx_ar_document_nbr.db
> INFO  [SASI-Memtable:1] 2016-11-03 21:00:19,450 PerSSTableIndexWriter.java:335 - Index
flush to /u01/cassandra-data/data/essatc1/audit_record-520c1dc0a1e411e691db0bd4b103bd15/mc-266-big-SI_idx_ar_document_nbr.db
took 33 ms.
> ERROR [SASI-Memtable:1] 2016-11-03 21:00:19,454 CassandraDaemon.java:229 - Exception
in thread Thread[SASI-Memtable:1,5,main]
> java.lang.AssertionError: cannot have more than 8 overflow collisions per leaf, but had:
25
> 	at org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder$Leaf.createOverflowEntry(AbstractTokenTreeBuilder.java:357)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder$Leaf.createEntry(AbstractTokenTreeBuilder.java:346)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.DynamicTokenTreeBuilder$DynamicLeaf.serializeData(DynamicTokenTreeBuilder.java:180)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder$Leaf.serialize(AbstractTokenTreeBuilder.java:306)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder.write(AbstractTokenTreeBuilder.java:90)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$MutableDataBlock.flushAndClear(OnDiskIndexBuilder.java:629)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$MutableLevel.flush(OnDiskIndexBuilder.java:446)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$MutableLevel.add(OnDiskIndexBuilder.java:433)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.addTerm(OnDiskIndexBuilder.java:207)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:293)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:258)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:241)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.lambda$scheduleSegmentFlush$0(PerSSTableIndexWriter.java:267)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.lambda$complete$1(PerSSTableIndexWriter.java:296)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_91]
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_91]
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_91]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91]
> 	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> ERROR [MemtableFlushWriter:5] 2016-11-03 21:00:19,459 DataTracker.java:168 - Can't open
index file at /u01/cassandra-data/data/essatc1/audit_record-520c1dc0a1e411e691db0bd4b103bd15/mc-266-big-SI_idx_ar_document_nbr.db,
skipping.
> java.lang.IllegalArgumentException: position: -3995747617649042944, limit: 20480
> 	at org.apache.cassandra.index.sasi.utils.MappedBuffer.position(MappedBuffer.java:106)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.disk.OnDiskIndex.<init>(OnDiskIndex.java:147)
~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.SSTableIndex.<init>(SSTableIndex.java:62) ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.conf.DataTracker.getIndexes(DataTracker.java:150)
[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.conf.DataTracker.update(DataTracker.java:69) [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.conf.ColumnIndex.update(ColumnIndex.java:147) [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.index.sasi.SASIIndex.handleNotification(SASIIndex.java:320)
[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.db.lifecycle.Tracker.notifyAdded(Tracker.java:421) [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.db.lifecycle.Tracker.replaceFlushed(Tracker.java:356) [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.db.compaction.CompactionStrategyManager.replaceFlushed(CompactionStrategyManager.java:317)
[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.db.ColumnFamilyStore.replaceFlushed(ColumnFamilyStore.java:1574)
[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1197)
[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1100)
[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91]
> 	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
> {quote}
> I can attach more stack traces, but it's just more of the same.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message