Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 10C07200BBA for ; Sat, 5 Nov 2016 16:21:42 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 0F4F4160AEF; Sat, 5 Nov 2016 15:21:42 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id B129C160AE0 for ; Sat, 5 Nov 2016 16:21:40 +0100 (CET) Received: (qmail 20998 invoked by uid 500); 5 Nov 2016 15:21:39 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 20988 invoked by uid 99); 5 Nov 2016 15:21:39 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 05 Nov 2016 15:21:39 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id AC31B1806ED for ; Sat, 5 Nov 2016 15:21:38 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.38 X-Spam-Level: ** X-Spam-Status: No, score=2.38 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id SbBkcdPSWmV5 for ; Sat, 5 Nov 2016 15:21:34 +0000 (UTC) Received: from mail-vk0-f51.google.com (mail-vk0-f51.google.com [209.85.213.51]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 192EE5F369 for ; Sat, 5 Nov 2016 15:21:34 +0000 (UTC) Received: by mail-vk0-f51.google.com with SMTP id w194so90028777vkw.2 for ; Sat, 05 Nov 2016 08:21:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Ay032v5otGGPZaClFNr7Z4cZ25GnUPR4jDUau/ovunU=; b=MPz/Z016ticxr7pYMfWeEZDdJb8sE8mWJZbRP6qFmei5uhK4JUF4IW5zJyXJQ7ilvK okyRMrvCopjl3zMby0KLdkukvNhdkFPL3LMTp86j4WK1I5xSiaPM04VQNDnu5HIO11c0 w69+UJu8EW0ALW13XKBFPJ2T7h3P8QUaSlssHK0+V5GH78kbHy4/jk3FeV6BHrfwkXc2 FB2nHK5tlg/tQA0mu1awR0KZ+DVn5/yXdIh/8zQrkPiLBbtrv6/e+JbsqpIdRt6uHZ8y LpoQDzbK8D1xxwr7e8BIgK0iksx8nmujok2t/LBuol4xN28sttZDZ3CPOOC0Cj52Sks+ HJjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Ay032v5otGGPZaClFNr7Z4cZ25GnUPR4jDUau/ovunU=; b=MC1pdAJUICkooxtvw79twmZEhP3o9hqdhGsQPrSA1A4T9AcH24pQNZW1JsWk9bnecg lOY7MyZpbFOxvBqHeRCC0/Wpbwu62G0Em7Y8khfU6/vkjxSe9xlVPj9gEn8kFYt0fK2P FfMgl1Cx0CnSkX+DJinuLN+RgF76kFeixk8ud/MpOGU7Oh27c6vt3IP0XHpI+xtaKs2U Q/fF0XOHVkYc9hEjYg1FSvmV4K5iT3AakMMIpKk8pR5HXjidcAKnQcFiYm7o8XqS++MM OlSRbDrSKyj2av7rWuAQIQBbV97MnVv3+0UNmY+2ZY4oOtfJPfkxXRmFfVd/sOOgPCnd jRTQ== X-Gm-Message-State: ABUngvdr0JsSqHudN7enBGQa3VhND7Y7Q0weTZzz1RFbED3DRL8AwIteKDijqoEGAOunBo56wkf8/wvHJfqcNA== X-Received: by 10.31.14.206 with SMTP id 197mr6139912vko.38.1478359293005; Sat, 05 Nov 2016 08:21:33 -0700 (PDT) MIME-Version: 1.0 Received: by 10.159.34.202 with HTTP; Sat, 5 Nov 2016 08:21:32 -0700 (PDT) In-Reply-To: References: From: Voytek Jarnot Date: Sat, 5 Nov 2016 10:21:32 -0500 Message-ID: Subject: Re: SASI index creation assertion error To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=001a11457f288295d705408f58e9 archived-at: Sat, 05 Nov 2016 15:21:42 -0000 --001a11457f288295d705408f58e9 Content-Type: text/plain; charset=UTF-8 Yep, already done: https://issues.apache.org/jira/browse/CASSANDRA-12877 On Fri, Nov 4, 2016 at 11:15 PM, Jonathan Haddad wrote: > Can you file a Jira for this? Would be good to make sure 3.10 doesn't get > released with this bug. > On Fri, Nov 4, 2016 at 6:11 PM Voytek Jarnot > wrote: > >> Thought I'd follow-up to myself, in case anyone else comes across this >> problem. I found a reasonably easy test case to reproduce the problem: >> >> This works in 3.9, but doesn't work in 3.10-snapshot: >> >> CREATE KEYSPACE vjtest WITH replication = {'class': 'SimpleStrategy', >> 'replication_factor': '1'}; >> use vjtest ; >> create table tester(id1 text, id2 text, id3 text, val1 text, primary >> key((id1, id2), id3)); >> create custom index tester_idx_val1 on tester(val1) using >> 'org.apache.cassandra.index.sasi.SASIIndex'; >> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','1-3','asdf'); >> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','2-3','asdf'); >> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','3-3','asdf'); >> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','4-3','asdf'); >> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','5-3','asdf'); >> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','6-3','asdf'); >> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','7-3','asdf'); >> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','8-3','asdf'); >> insert into tester(id1,id2,id3, val1) values ('1-1','1-2','9-3','asdf'); >> >> That's it - when Cassandra tries to flush all hell breaks loose (well, >> maaybe not, but an unhandled error gets logged). Also, the index doesn't >> actually work subsequently. >> >> On Fri, Nov 4, 2016 at 3:58 PM, Voytek Jarnot >> wrote: >> >> Wondering if anyone has encountered the same... >> >> Full story and stacktraces below, short version is that creating a SASI >> index fails for me when running a 3.10-SNAPSHOT build. One caveat: creating >> the index on an empty table doesn't fail; however, soon after I start >> pumping data into the table similar problems occur. >> >> I created CASSANDRA-12877 for this, but am beginning to suspect it might >> be related to CASSANDRA-11990. The thing that's throwing me is that I >> can't seem to duplicate this with a simple test table. >> >> Background: >> >> Ended up building/loading a 3.10-SNAPSHOT to try to get past >> CASSANDRA-11670, CASSANDRA-12223, and CASSANDRA-12689. >> >> 1) built/installed 3.10-SNAPSHOT from git branch cassandra-3.X >> 2) created keyspace (SimpleStrategy, RF 1) >> 3) created table: (simplified version below, many more valX columns >> present) >> >> CREATE TABLE test_table ( >> id1 text, >> id2 text, >> id3 date, >> id4 timestamp, >> id5 text, >> val1 text, >> val2 text, >> val3 text, >> task_id text, >> val4 text, >> val5 text, >> PRIMARY KEY ((id1, id2), id3, id4, id5) >> ) WITH CLUSTERING ORDER BY (id3 DESC, id4 DESC, id5 ASC) >> >> 4) created materialized view: >> >> CREATE MATERIALIZED VIEW test_table_by_task_id AS >> SELECT * >> FROM test_table >> WHERE id1 IS NOT NULL AND id2 IS NOT NULL AND id3 IS NOT NULL AND id4 >> IS NOT NULL AND id5 IS NOT NULL AND task_id IS NOT NULL >> PRIMARY KEY (task_id, id3, id4, id1, id2, id5) >> WITH CLUSTERING ORDER BY (id3 DESC, id4 DESC, id1 ASC, id2 ASC, id5 >> ASC) >> >> 5) inserted 27 million "rows" (i.e., unique values for id5) >> 6) create index attempt >> >> create custom index idx_test_table_val5 on test_table(val5) using >> 'org.apache.cassandra.index.sasi.SASIIndex'; >> >> 7) no error in cqlsh, but system.log shows many of the following: >> >> INFO [SASI-General:1] 2016-11-04 13:46:47,578 >> PerSSTableIndexWriter.java:277 - Flushed index segment >> /mydir/cassandra/apache-cassandra-3.10-SNAPSHOT/data/ >> data/mykeyspace/test_table-133dd090a2b411e6b1bf6df2a1af06 >> f0/mc-149-big-SI_idx_test_table_val5.db_0, took 869 ms. >> ERROR [SASI-General:1] 2016-11-04 13:46:47,584 CassandraDaemon.java:229 - >> Exception in thread Thread[SASI-General:1,5,main] >> java.lang.AssertionError: cannot have more than 8 overflow collisions per >> leaf, but had: 12 >> at org.apache.cassandra.index.sasi.disk. >> AbstractTokenTreeBuilder$Leaf.createOverflowEntry( >> AbstractTokenTreeBuilder.java:357) ~[apache-cassandra-3.10- >> SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.disk. >> AbstractTokenTreeBuilder$Leaf.createEntry(AbstractTokenTreeBuilder.java:346) >> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.disk.DynamicTokenTreeBuilder$ >> DynamicLeaf.serializeData(DynamicTokenTreeBuilder.java:180) >> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.disk. >> AbstractTokenTreeBuilder$Leaf.serialize(AbstractTokenTreeBuilder.java:306) >> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder. >> write(AbstractTokenTreeBuilder.java:90) ~[apache-cassandra-3.10- >> SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$ >> MutableDataBlock.flushAndClear(OnDiskIndexBuilder.java:629) >> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$ >> MutableLevel.flush(OnDiskIndexBuilder.java:446) ~[apache-cassandra-3.10- >> SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$ >> MutableLevel.add(OnDiskIndexBuilder.java:433) ~[apache-cassandra-3.10- >> SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder. >> addTerm(OnDiskIndexBuilder.java:207) ~[apache-cassandra-3.10- >> SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder. >> finish(OnDiskIndexBuilder.java:293) ~[apache-cassandra-3.10- >> SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder. >> finish(OnDiskIndexBuilder.java:258) ~[apache-cassandra-3.10- >> SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder. >> finish(OnDiskIndexBuilder.java:241) ~[apache-cassandra-3.10- >> SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.disk.PerSSTableIndexWriter$Index. >> lambda$scheduleSegmentFlush$0(PerSSTableIndexWriter.java:267) >> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT] >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> ~[na:1.8.0_101] >> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> ~[na:1.8.0_101] >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> [na:1.8.0_101] >> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101] >> >> As well as some of these: >> >> ERROR [CompactionExecutor:3] 2016-11-04 13:49:13,142 DataTracker.java:168 >> - Can't open index file at /mydir/cassandra/apache- >> cassandra-3.10-SNAPSHOT/data/data/mykeyspace/test_table- >> 133dd090a2b411e6b1bf6df2a1af06f0/mc-300-big-SI_idx_test_table_val5.db, >> skipping. >> java.lang.IllegalArgumentException: position: 3472329188772431788, >> limit: 8180 >> at org.apache.cassandra.index.sasi.utils.MappedBuffer. >> position(MappedBuffer.java:106) ~[apache-cassandra-3.10- >> SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.disk.OnDiskIndex.(OnDiskIndex.java:147) >> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.SSTableIndex.(SSTableIndex.java:62) >> ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.conf.DataTracker. >> getIndexes(DataTracker.java:150) [apache-cassandra-3.10- >> SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.conf.DataTracker.update(DataTracker.java:69) >> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.conf.ColumnIndex.update(ColumnIndex.java:147) >> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.SASIIndexBuilder.completeSSTable(SASIIndexBuilder.java:156) >> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.index.sasi.SASIIndexBuilder.build(SASIIndexBuilder.java:125) >> [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT] >> at org.apache.cassandra.db.compaction.CompactionManager$ >> 14.run(CompactionManager.java:1583) [apache-cassandra-3.10- >> SNAPSHOT.jar:3.10-SNAPSHOT] >> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> [na:1.8.0_101] >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> [na:1.8.0_101] >> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> [na:1.8.0_101] >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> [na:1.8.0_101] >> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> [na:1.8.0_101] >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> [na:1.8.0_101] >> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101] >> >> >> --001a11457f288295d705408f58e9 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Fri, Nov 4, 2016 at 11:15 PM, Jonathan Haddad <<= a href=3D"mailto:jon@jonhaddad.com" target=3D"_blank">jon@jonhaddad.com= > wrote:
Can you file a Jira fo= r this? Would be good to make sure 3.10 doesn't get released with this = bug.
On Fri, Nov 4, 2016 at 6:11 PM Voytek Jarnot <voytek.jarnot@gmail.c= om> wrote:
Thought I'd follow-up to myself, in case anyone else come= s across this problem.=C2=A0 I found a reasonably easy test case to reprodu= ce the problem:

This works i= n 3.9, but doesn't work in 3.10-snapshot:

CREATE KEYSPACE vjtest WITH replica= tion =3D {'class': 'SimpleStrategy', 'replication_facto= r': '1'};
u= se vjtest ;
create tabl= e tester(id1 text, id2 text, id3 text, val1 text, primary key((id1, id2), i= d3));
create custom ind= ex tester_idx_val1 on tester(val1) using 'org.apache.cassandra.index.sasi.SASIIndex';
insert into tester(id1,id2,id3, val1) values ('1-1','1-2',= '1-3','asdf');
insert into tester(id1,id2,id3, val1) values ('1-1','1-= 2','2-3','asdf');
insert into tester(id1,id2,id3, val1) values ('1-1',= '1-2','3-3','asdf');
insert into tester(id1,id2,id3, val1) values ('1-= 1','1-2','4-3','asdf');
insert into tester(id1,id2,id3, val1) values (= '1-1','1-2','5-3','asdf');
insert into tester(id1,id2,id3, val1) = values ('1-1','1-2','6-3','asdf');
insert into tester(id1,id2,id3,= val1) values ('1-1','1-2','7-3','asdf');
insert into tester(id1,i= d2,id3, val1) values ('1-1','1-2','8-3','asdf&#= 39;);
insert into teste= r(id1,id2,id3, val1) values ('1-1','1-2','9-3','= ;asdf');

That's it - when Cassandra tries to flush all hell break= s loose (well, maaybe not, but an unhandled error gets logged).=C2=A0 Also,= the index doesn't actually work subsequently.

On Fri, Nov 4, 2016 at 3:58 PM, Voytek Jarnot <voytek.jarn= ot@gmail.com> wrote:
Wondering if anyone has encountered the same... =C2= =A0

Full story and stacktraces below, short version is that creating a SAS= I index fails for me when running a 3.10-SNAPSHOT build. One caveat: creati= ng the index on an empty table doesn't fail; however, soon after I star= t pumping data into the table similar problems occur.

I created CASSANDRA-= 12877 for this, but am beginning to suspect it might be related to CASSANDR= A-11990.=C2=A0 The thing that's throwing me is that I can't seem to= duplicate this with a simple test table.

Background:

Ended up building/lo= ading a 3.10-SNAPSHOT to try to get past CASSANDRA-11670,=C2=A0CASSANDRA-12223, and CASSANDRA-12689.

1) built/installed 3.10-SNAPSHOT from git branch = cassandra-3.X
2) create= d keyspace (SimpleStrategy, RF 1)
3) created table: (simplified version below, many more valX colu= mns present)
C= REATE TABLE test_table (
=C2=A0 =C2=A0 id1 text,
=C2=A0 =C2=A0 id2 text,
=C2=A0 =C2=A0 id3 date,
=C2=A0 =C2=A0 id4 timestamp,
=C2=A0 =C2=A0 id5 text,
=C2=A0 =C2=A0 val1 text,
=C2=A0 =C2=A0 val2 text,
=C2=A0 =C2=A0 val3 text,
=C2=A0 =C2=A0 task_id text,
=C2=A0 =C2=A0 val4 text,
=C2=A0 =C2=A0 val5 text,
=C2=A0 =C2=A0 PRIMARY KEY ((id1, id2), id3, id4,= id5)
) WITH CLUSTERING ORDER BY (id3 DESC, id4 DESC= , id5 ASC)
4) created materialized view:
<= div class=3D"m_-5494366931265731481gmail_msg">
CREATE MATERIALIZED VIEW test_table_by_task_id AS
=C2=A0 =C2=A0 SELECT *
=C2=A0 =C2=A0 FROM test_table<= /div>
=C2=A0 =C2=A0 WHERE id1= IS NOT NULL AND id2 IS NOT NULL AND id3 IS NOT NULL AND id4 IS NOT NULL AN= D id5 IS NOT NULL AND task_id IS NOT NULL
=C2=A0 =C2=A0 PRIMARY KEY (task_id, id3, id4, id1, id2, = id5)
=C2=A0 =C2=A0 WITH= CLUSTERING ORDER BY (id3 DESC, id4 DESC, id1 ASC, id2 ASC, id5 ASC)
<= /div>
5) inserted 27 million "rows" (i.e., unique val= ues for id5)
6) create = index attempt
= create custom index idx_test_table_val5 on test_table(val5) using 'org.= apache.cassandra.index.sasi.SASIIndex';
7) no error i= n cqlsh, but system.log shows many of the following:

INFO =C2=A0[SASI-General:1] 2016-1= 1-04 13:46:47,578 PerSSTableIndexWriter.java:277 - Flushed index segment /m= ydir/cassandra/apache-cassandra-3.10-SNAPSHOT/data/data/mykeyspac= e/test_table-133dd090a2b411e6b1bf6df2a1af06f0/mc-149-big-SI_idx_t= est_table_val5.db_0, took 869 ms.
ERROR [SASI-General:1] 2016-11-04 13:46:47,584 CassandraDae= mon.java:229 - Exception in thread Thread[SASI-General:1,5,main]
java.lang.AssertionError: cannot = have more than 8 overflow collisions per leaf, but had: 12
=C2=A0 =C2=A0 at org.apache.cassandra.= index.sasi.disk.AbstractTokenTreeBuilder$Leaf.createOverflow= Entry(AbstractTokenTreeBuilder.java:357) ~[apache-cassandra-3.10-= SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache.cassandra.index.sasi.disk.A= bstractTokenTreeBuilder$Leaf.createEntry(AbstractTokenTreeBuilder= .java:346) ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apac= he.cassandra.index.sasi.disk.DynamicTokenTreeBuilder$Dynamic= Leaf.serializeData(DynamicTokenTreeBuilder.java:180) ~[apache-cas= sandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache.cassandra.index.sasi= .disk.AbstractTokenTreeBuilder$Leaf.serialize(AbstractTokenT= reeBuilder.java:306) ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SN= APSHOT]
=C2=A0 =C2=A0 a= t org.apache.cassandra.index.sasi.disk.AbstractTokenTreeBuilder.<= wbr>write(AbstractTokenTreeBuilder.java:90) ~[apache-cassandra-3.= 10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache.cassandra.index.sasi.disk.OnD= iskIndexBuilder$MutableDataBlock.flushAndClear(OnDiskIndexBu= ilder.java:629) ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apac= he.cassandra.index.sasi.disk.OnDiskIndexBuilder$MutableLevel.flus= h(OnDiskIndexBuilder.java:446) ~[apache-cassandra-3.10-SNAPSHOT.j= ar:3.10-SNAPSHOT]
=C2= =A0 =C2=A0 at org.apache.cassandra.index.sasi.disk.OnDiskIndexBuilder$= MutableLevel.add(OnDiskIndexBuilder.java:433) ~[apache-cassandra-= 3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache.cassandra.index.sasi.disk.O= nDiskIndexBuilder.addTerm(OnDiskIndexBuilder.java:207) ~[apache-c= assandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache.cassandra.index.sa= si.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java:293) ~[= apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache.cassandra.index= .sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.java= :258) ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache.cassand= ra.index.sasi.disk.OnDiskIndexBuilder.finish(OnDiskIndexBuilder.<= wbr>java:241) ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache= .cassandra.index.sasi.disk.PerSSTableIndexWriter$Index.lambd= a$scheduleSegmentFlush$0(PerSSTableIndexWriter.java:267) ~[apache= -cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at java.util.concurrent.Future= Task.run(FutureTask.java:266) ~[na:1.8.0_101]
=C2=A0 =C2=A0 at java.util.concurrent.Thre= adPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_101]=
=C2=A0 =C2=A0 at java.= util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.= java:617) [na:1.8.0_101]
=C2=A0 =C2=A0 at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101= ]

As well as some of these:

ERROR [CompactionExecutor:3] 2016-11-04 13:49:13,142= DataTracker.java:168 - Can't open index file at /mydir/cassandra/apach= e-cassandra-3.10-SNAPSHOT/data/data/mykeyspace/test_table-13= 3dd090a2b411e6b1bf6df2a1af06f0/mc-300-big-SI_idx_test_table_val5.= db, skipping.
java.lang= .IllegalArgumentException: position: 3472329188772431788, limit: 8180<= /div>
=C2=A0 =C2=A0 at org.ap= ache.cassandra.index.sasi.utils.MappedBuffer.position(MappedBuffe= r.java:106) ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apa= che.cassandra.index.sasi.disk.OnDiskIndex.<init>(OnDiskInde= x.java:147) ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
<= div class=3D"m_-5494366931265731481gmail_msg">=C2=A0 =C2=A0 at org.apache.c= assandra.index.sasi.SSTableIndex.<init>(SSTableIndex.java:6= 2) ~[apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache.cassandra.= index.sasi.conf.DataTracker.getIndexes(DataTracker.java:150)= [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache.cassandra.ind= ex.sasi.conf.DataTracker.update(DataTracker.java:69) [apache-cass= andra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache.cassandra.index.sasi.= conf.ColumnIndex.update(ColumnIndex.java:147) [apache-cassandra-3.10-<= wbr>SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache.cassandra.index.sasi.SASIIndexBui= lder.completeSSTable(SASIIndexBuilder.java:156) [apache-cassandra= -3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache.cassandra.index.sasi.SASII= ndexBuilder.build(SASIIndexBuilder.java:125) [apache-cassandra-3.10-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at org.apache.cassandra.db.compaction.Compaction= Manager$14.run(CompactionManager.java:1583) [apache-cassandra-3.1= 0-SNAPSHOT.jar:3.10-SNAPSHOT]
=C2=A0 =C2=A0 at java.util.concurrent.Executors$RunnableAd= apter.call(Executors.java:511) [na:1.8.0_101]
=C2=A0 =C2=A0 at java.util.concurrent.Futu= reTask.run(FutureTask.java:266) [na:1.8.0_101]
=C2=A0 =C2=A0 at java.util.concurrent.Exe= cutors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_101]
=C2=A0 =C2=A0 at java.util.con= current.FutureTask.run(FutureTask.java:266) [na:1.8.0_101]
<= div class=3D"m_-5494366931265731481gmail_msg">=C2=A0 =C2=A0 at java.util.co= ncurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:114= 2) [na:1.8.0_101]
=C2= =A0 =C2=A0 at java.util.concurrent.ThreadPoolExecutor$Worker.run(= ThreadPoolExecutor.java:617) [na:1.8.0_101]
=C2=A0 =C2=A0 at java.lang.Thread.run(Thread.java= :745) [na:1.8.0_101]



--001a11457f288295d705408f58e9--