cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ariel Weisberg (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-10778) CompactionExecutor writing to old index files
Date Mon, 30 Nov 2015 18:42:11 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15032232#comment-15032232
] 

Ariel Weisberg commented on CASSANDRA-10778:
--------------------------------------------

[The original logic was that people are expected to run {{upgradesstables}}|https://issues.apache.org/jira/browse/CASSANDRA-9738?focusedCommentId=14741107&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14741107]
when upgrading Cassandra so we didn't add support for serializing the old format.

{quote}
Jonathan told me the expectation is that people run upgrade sstables so we don't need to be
heroic. Let's go for the simples possible solution which is making the old and new formats
match after deserialization. Hopefully this means we can remove a bunch of paths based in
which format we are looking at.
{quote}

Hitting an assertion when it is expected that there will be a period before {{upgradesstables}}
is run doesn't make sense. I think what we want to do instead is skip index entries that are
in the old format and just not write them. Newer entries will still get written so you won't
have a completely cold cache and if the access pattern is temporal most things in the cache
will be newer entries anyways.

[~slebresne] does it make sense to you that since people will run {{upgradesstables}} there
is no need to implement write path serialization (and tests) for the old format?

> CompactionExecutor writing to old index files
> ---------------------------------------------
>
>                 Key: CASSANDRA-10778
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10778
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Local Write-Read Paths
>         Environment: Windows 7 64-bit, Cassandra 3.0.0, Java 1.8u60
>            Reporter: Will Zhang
>            Assignee: Ariel Weisberg
>              Labels: error
>             Fix For: 3.0.1, 3.1
>
>
> Hi,
> I have been running some test for upgrading from v2.2.2 to v3.0.0. 
> I encountered the following `ERROR` in the `system.log` when I was *creating/dropping
materialized views*. I did some searches online but couldn't find anything useful so filing
this. The log seem to suggest that it is bug-like.
> Any thoughts on this would be appreciated.
> Main error line in log:
> {code}
> ERROR [CompactionExecutor:4] 2015-11-26 15:40:56,033 CassandraDaemon.java:195 - Exception
in thread Thread[CompactionExecutor:4,1,main]
>     java.lang.AssertionError: We read old index files but we should never write them
> {code}
> Longer log:
> {code}
>     INFO  [SharedPool-Worker-2] 2015-11-26 15:25:37,152 MigrationManager.java:336 - Create
new view: org.apache.cassandra.config.ViewDefinition@1b7fc5e6[ksName=demo,viewName=broker_quotes_by_date,baseTableId=bf928280-3c23-11e5-a4ba-07dc7eba8ee2,baseTableName=broker_quotes,includeAllColumns=true,whereClause=date
IS NOT NULL AND datetime IS NOT NULL AND isin IS NOT NULL AND side IS NOT NULL AND broker
IS NOT NULL,metadata=org.apache.cassandra.config.CFMetaData@49123522[cfId=f19cb8f0-9451-11e5-af90-6916ca23ea25,ksName=demo,cfName=broker_quotes_by_date,flags=[COMPOUND],params=TableParams{comment=,
read_repair_chance=0.0, dclocal_read_repair_chance=0.1, bloom_filter_fp_chance=0.01, crc_check_chance=1.0,
gc_grace_seconds=864000, default_time_to_live=0, memtable_flush_period_in_ms=0, min_index_interval=128,
max_index_interval=2048, speculative_retry=99PERCENTILE, caching={'keys' : 'ALL', 'rows_per_partition'
: 'NONE'}, compaction=CompactionParams{class=org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy,
options={min_threshold=4, max_threshold=32}}, compression=org.apache.cassandra.schema.CompressionParams@f3ef4959,
extensions={}},comparator=comparator(org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.TimestampType),
org.apache.cassandra.db.marshal.UTF8Type, org.apache.cassandra.db.marshal.UTF8Type, org.apache.cassandra.db.marshal.UTF8Type),partitionColumns=[[]
| [bmark_spread g_spread is_axed oas_spread price size ytw z_spread]],partitionKeyColumns=[ColumnDefinition{name=date,
type=org.apache.cassandra.db.marshal.TimestampType, kind=PARTITION_KEY, position=0}],clusteringColumns=[ColumnDefinition{name=datetime,
type=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.TimestampType),
kind=CLUSTERING, position=0}, ColumnDefinition{name=side, type=org.apache.cassandra.db.marshal.UTF8Type,
kind=CLUSTERING, position=1}, ColumnDefinition{name=isin, type=org.apache.cassandra.db.marshal.UTF8Type,
kind=CLUSTERING, position=2}, ColumnDefinition{name=broker, type=org.apache.cassandra.db.marshal.UTF8Type,
kind=CLUSTERING, position=3}],keyValidator=org.apache.cassandra.db.marshal.TimestampType,columnMetadata=[ColumnDefinition{name=z_spread,
type=org.apache.cassandra.db.marshal.FloatType, kind=REGULAR, position=-1}, ColumnDefinition{name=datetime,
type=org.apache.cassandra.db.marshal.ReversedType(org.apache.cassandra.db.marshal.TimestampType),
kind=CLUSTERING, position=0}, ColumnDefinition{name=date, type=org.apache.cassandra.db.marshal.TimestampType,
kind=PARTITION_KEY, position=0}, ColumnDefinition{name=oas_spread, type=org.apache.cassandra.db.marshal.FloatType,
kind=REGULAR, position=-1}, ColumnDefinition{name=isin, type=org.apache.cassandra.db.marshal.UTF8Type,
kind=CLUSTERING, position=2}, ColumnDefinition{name=bmark_spread, type=org.apache.cassandra.db.marshal.FloatType,
kind=REGULAR, position=-1}, ColumnDefinition{name=side, type=org.apache.cassandra.db.marshal.UTF8Type,
kind=CLUSTERING, position=1}, ColumnDefinition{name=broker, type=org.apache.cassandra.db.marshal.UTF8Type,
kind=CLUSTERING, position=3}, ColumnDefinition{name=is_axed, type=org.apache.cassandra.db.marshal.BooleanType,
kind=REGULAR, position=-1}, ColumnDefinition{name=ytw, type=org.apache.cassandra.db.marshal.FloatType,
kind=REGULAR, position=-1}, ColumnDefinition{name=price, type=org.apache.cassandra.db.marshal.FloatType,
kind=REGULAR, position=-1}, ColumnDefinition{name=g_spread, type=org.apache.cassandra.db.marshal.FloatType,
kind=REGULAR, position=-1}, ColumnDefinition{name=size, type=org.apache.cassandra.db.marshal.DoubleType,
kind=REGULAR, position=-1}],droppedColumns={},triggers=[],indexes=[]]]
>     INFO  [MigrationStage:1] 2015-11-26 15:25:37,440 ColumnFamilyStore.java:381 - Initializing
demo.broker_quotes_by_date
>     ERROR [CompactionExecutor:4] 2015-11-26 15:40:56,033 CassandraDaemon.java:195 - Exception
in thread Thread[CompactionExecutor:4,1,main]
>     java.lang.AssertionError: We read old index files but we should never write them
> 	    at org.apache.cassandra.db.RowIndexEntry$Serializer.serialize(RowIndexEntry.java:130)
~[apache-cassandra-3.0.0.jar:3.0.0]
> 	    at org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:466)
~[apache-cassandra-3.0.0.jar:3.0.0]
> 	    at org.apache.cassandra.service.CacheService$KeyCacheSerializer.serialize(CacheService.java:454)
~[apache-cassandra-3.0.0.jar:3.0.0]
> 	    at org.apache.cassandra.cache.AutoSavingCache$Writer.saveCache(AutoSavingCache.java:358)
~[apache-cassandra-3.0.0.jar:3.0.0]
> 	    at org.apache.cassandra.db.compaction.CompactionManager$12.run(CompactionManager.java:1302)
~[apache-cassandra-3.0.0.jar:3.0.0]
> 	    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_60]
> 	    at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_60]
> 	    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_60]
> 	    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_60]
> 	    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
> 	INFO  [IndexSummaryManager:1] 2015-11-26 15:41:10,729 IndexSummaryManager.java:257 -
Redistributing index summaries
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message