cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ludovic Boutros (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-13403) nodetool repair breaks SASI index
Date Tue, 07 Nov 2017 14:50:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-13403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16242120#comment-16242120
] 

Ludovic Boutros commented on CASSANDRA-13403:
---------------------------------------------

And the if we rebuild the index:

{code}
INFO  [RMI TCP Connection(7)-10.53.0.15] 2017-11-07 15:44:34,456 ColumnFamilyStore.java:806
- User Requested secondary index re-build for lubo_test/t_doc indexes: i_doc
DEBUG [RMI TCP Connection(7)-10.53.0.15] 2017-11-07 15:44:34,458 ColumnFamilyStore.java:899
- Enqueuing flush of IndexInfo: 0,385KiB (0%) on-heap, 0,000KiB (0%) off-heap
DEBUG [PerDiskMemtableFlushWriter_0:5] 2017-11-07 15:44:34,514 Memtable.java:461 - Writing
Memtable-IndexInfo@1020363412(0,049KiB serialized bytes, 1 ops, 0%/0% of on/off-heap limit),
flushed range = (min(-9223372036854775808), max(9223372036854775807)]
DEBUG [PerDiskMemtableFlushWriter_0:5] 2017-11-07 15:44:34,515 Memtable.java:490 - Completed
flushing /data/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/mc-18-big-Data.db
(0,036KiB) for commitlog position CommitLogPosition(segmentId=1510062526702, position=2214781)
DEBUG [MemtableFlushWriter:5] 2017-11-07 15:44:34,644 ColumnFamilyStore.java:1197 - Flushed
to [BigTableReader(path='/data/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/mc-18-big-Data.db')]
(1 sstables, 4,854KiB), biggest 4,854KiB, smallest 4,854KiB
INFO  [RMI TCP Connection(7)-10.53.0.15] 2017-11-07 15:44:34,644 SecondaryIndexManager.java:365
- Submitting index build of i_doc for data in BigTableReader(path='/data/cassandra/data/lubo_test/t_doc-64343790c31611e7a46403e2ed27ae86/mc-23-big-Data.db'),BigTableReader(path='/data/cassandra/data/lubo_test/t_doc-64343790c31611e7a46403e2ed27ae86/mc-22-big-Data.db')
INFO  [CompactionExecutor:10] 2017-11-07 15:44:34,646 PerSSTableIndexWriter.java:279 - Scheduling
index flush to /data/cassandra/data/lubo_test/t_doc-64343790c31611e7a46403e2ed27ae86/mc-22-big-SI_i_doc.db
INFO  [SASI-General:3] 2017-11-07 15:44:34,675 PerSSTableIndexWriter.java:330 - Index flush
to /data/cassandra/data/lubo_test/t_doc-64343790c31611e7a46403e2ed27ae86/mc-22-big-SI_i_doc.db
took 28 ms.
{code}
{code}
INFO  [CompactionExecutor:10] 2017-11-07 15:44:34,676 DataTracker.java:152 - SSTableIndex.open(column:
r, minTerm: 0, maxTerm: 0, minKey: 1, maxKey: 7, sstable: BigTableReader(path='/data/cassandra/data/lubo_test/t_doc-64343790c31611e7a46403e2ed27ae86/mc-22-big-Data.db'))
{code}
{code}
INFO  [CompactionExecutor:10] 2017-11-07 15:44:34,677 PerSSTableIndexWriter.java:279 - Scheduling
index flush to /data/cassandra/data/lubo_test/t_doc-64343790c31611e7a46403e2ed27ae86/mc-23-big-SI_i_doc.db
INFO  [SASI-General:3] 2017-11-07 15:44:34,683 PerSSTableIndexWriter.java:330 - Index flush
to /data/cassandra/data/lubo_test/t_doc-64343790c31611e7a46403e2ed27ae86/mc-23-big-SI_i_doc.db
took 5 ms.
{code}
{code}
INFO  [CompactionExecutor:10] 2017-11-07 15:44:34,683 DataTracker.java:152 - SSTableIndex.open(column:
r, minTerm: 0, maxTerm: 0, minKey: 11, maxKey: 9, sstable: BigTableReader(path='/data/cassandra/data/lubo_test/t_doc-64343790c31611e7a46403e2ed27ae86/mc-23-big-Data.db'))
{code}
{code}
INFO  [RMI TCP Connection(7)-10.53.0.15] 2017-11-07 15:44:34,683 SecondaryIndexManager.java:385
- Index build of i_doc complete
{code}

We can see the two lines of log of the DataTracker.



> nodetool repair breaks SASI index
> ---------------------------------
>
>                 Key: CASSANDRA-13403
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13403
>             Project: Cassandra
>          Issue Type: Bug
>          Components: sasi
>         Environment: 3.10
>            Reporter: Igor Novgorodov
>            Assignee: Alex Petrov
>         Attachments: 3_nodes_compaction.log, 4_nodes_compaction.log
>
>
> I've got table:
> {code}
> CREATE TABLE cservice.bulks_recipients (
>     recipient text,
>     bulk_id uuid,
>     datetime_final timestamp,
>     datetime_sent timestamp,
>     request_id uuid,
>     status int,
>     PRIMARY KEY (recipient, bulk_id)
> ) WITH CLUSTERING ORDER BY (bulk_id ASC)
>     AND bloom_filter_fp_chance = 0.01
>     AND caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'}
>     AND comment = ''
>     AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
>     AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
>     AND crc_check_chance = 1.0
>     AND dclocal_read_repair_chance = 0.1
>     AND default_time_to_live = 0
>     AND gc_grace_seconds = 864000
>     AND max_index_interval = 2048
>     AND memtable_flush_period_in_ms = 0
>     AND min_index_interval = 128
>     AND read_repair_chance = 0.0
>     AND speculative_retry = '99PERCENTILE';
> CREATE CUSTOM INDEX bulk_recipients_bulk_id ON cservice.bulks_recipients (bulk_id) USING
'org.apache.cassandra.index.sasi.SASIIndex';
> {code}
> There are 11 rows in it:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> Let's query by index (all rows have the same *bulk_id*):
> {code}
> > select * from bulks_recipients where bulk_id = baa94815-e276-4ca4-adda-5b9734e6c4a5;
                                                
> ...
> (11 rows)
> {code}
> Ok, everything is fine.
> Now i'm doing *nodetool repair --partitioner-range --job-threads 4 --full* on each node
in cluster sequentially.
> After it finished:
> {code}
> > select * from bulks_recipients where bulk_id = baa94815-e276-4ca4-adda-5b9734e6c4a5;
> ...
> (2 rows)
> {code}
> Only two rows.
> While the rows are actually there:
> {code}
> > select * from bulks_recipients;
> ...
> (11 rows)
> {code}
> If i issue an incremental repair on a random node, i can get like 7 rows after index
query.
> Dropping index and recreating it fixes the issue. Is it a bug or am i doing the repair
the wrong way?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message