cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kurt Greaves (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-13565) Materialized view usage of commit logs requires large mutation but commitlog_segment_size_in_mb=2048 causes exception
Date Wed, 28 Jun 2017 03:55:00 GMT


Kurt Greaves commented on CASSANDRA-13565:

Certainly covered under CASSANDRA-13622. If anyone would like to fix it in this case fixing
the overflow should be enough. Under either ticket is reasonable.

> Materialized view usage of commit logs requires large mutation but commitlog_segment_size_in_mb=2048
causes exception
> ---------------------------------------------------------------------------------------------------------------------
>                 Key: CASSANDRA-13565
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Configuration, Materialized Views, Streaming and Messaging
>         Environment: Cassandra 3.9.0, Windows 
>            Reporter: Tania S Engel
>         Attachments: CQLforTable.png
> We will be upgrading to 3.10 for CASSANDRA-11670. However, there is another scenario
(not applyunsafe during JOIN) which leads to :
> 	java.lang.IllegalArgumentException: Mutation of 525.847MiB is too large for the maximum
size of 512.000MiB
> 	            at org.apache.cassandra.db.commitlog.CommitLog.add(
> 	            at org.apache.cassandra.db.Keyspace.apply(
> 	            at org.apache.cassandra.db.Keyspace.apply(
> 	            at org.apache.cassandra.db.Mutation.applyFuture(
> 	            at org.apache.cassandra.db.Mutation.apply(
> 	            at
> 	            at org.apache.cassandra.service.StorageProxy.mutateMV(
> 	            at org.apache.cassandra.db.view.ViewBuilder.buildKey(
> 	            at
> 	            at org.apache.cassandra.db.compaction.CompactionManager$
> 	            at java.util.concurrent.Executors$
> 	            at [na:1.8.0_66]
> 	            at java.util.concurrent.ThreadPoolExecutor.runWorker(
> 	            at java.util.concurrent.ThreadPoolExecutor$
> 	            at [na:1.8.0_66] 
> Due to the relationship of max_mutation_size_in_kb and commitlog_segment_size_in_mb,
we increased commitlog_segment_size_in_mb and left Cassandra to calculate max_mutation_size_in_kb
as half the size commitlog_segment_size_in_mb * 1024.
>  However, we have found that if we set commitlog_segment_size_in_mb=2048 we get an exception
upon starting Cassandra, when it is creating a new commit log.
> ERROR [COMMIT-LOG-ALLOCATOR] 2017-05-31 17:01:48,005 -
Exiting due to error while processing commit log during initialization.
> An attempt was made to move
the file pointer before the beginning of the file
> Perhaps the index you are using is not big enough and it goes negative.
> Is the relationship between max_mutation_size_in_kb and commitlog_segment_size_in_mb
important to preserve? In our limited stress test we are finding mutation size already over
512mb and we expect more data in our sstables and associated materialized views.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message