cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tania S Engel (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CASSANDRA-13565) Materialized view usage of commit logs requires large mutation but commitlog_segment_size_in_mb=2048 causes exception
Date Wed, 31 May 2017 17:07:04 GMT
Tania S Engel created CASSANDRA-13565:
-----------------------------------------

             Summary: Materialized view usage of commit logs requires large mutation but commitlog_segment_size_in_mb=2048
causes exception
                 Key: CASSANDRA-13565
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13565
             Project: Cassandra
          Issue Type: Bug
          Components: Configuration, Materialized Views, Streaming and Messaging
         Environment: Cassandra 3.9.0, Windows 
            Reporter: Tania S Engel


We will be upgrading to 3.10 for CASSANDRA-11670. However, there is another scenario (not
applyunsafe during JOIN) which leads to :
	java.lang.IllegalArgumentException: Mutation of 525.847MiB is too large for the maximum size
of 512.000MiB
	            at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:262)
~[apache-cassandra-3.9.0.jar:3.9.0]
	            at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:493) ~[apache-cassandra-3.9.0.jar:3.9.0]
	            at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:396) ~[apache-cassandra-3.9.0.jar:3.9.0]
	            at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:215)
~[apache-cassandra-3.9.0.jar:3.9.0]
	            at org.apache.cassandra.db.Mutation.apply(Mutation.java:227) ~[apache-cassandra-3.9.0.jar:3.9.0]
	            at org.apache.cassandra.batchlog.BatchlogManager.store(BatchlogManager.java:147)
~[apache-cassandra-3.9.0.jar:3.9.0]
	            at org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:797)
~[apache-cassandra-3.9.0.jar:3.9.0]
	            at org.apache.cassandra.db.view.ViewBuilder.buildKey(ViewBuilder.java:96)
~[apache-cassandra-3.9.0.jar:3.9.0]
	            at org.apache.cassandra.db.view.ViewBuilder.run(ViewBuilder.java:165)
~[apache-cassandra-3.9.0.jar:3.9.0]
	            at org.apache.cassandra.db.compaction.CompactionManager$14.run(CompactionManager.java:1591)
[apache-cassandra-3.9.0.jar:3.9.0]
	            at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_66]
	            at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_66]
	            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_66]
	            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_66]
	            at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66] 

Due to the relationship of max_mutation_size_in_kb and commitlog_segment_size_in_mb, we increased
commitlog_segment_size_in_mb and left Cassandra to calculate max_mutation_size_in_kb as half
the size commitlog_segment_size_in_mb * 1024.

 However, we have found that if we set commitlog_segment_size_in_mb=2048 we get an exception
upon starting Cassandra, when it is creating a new commit log.

ERROR [COMMIT-LOG-ALLOCATOR] 2017-05-31 17:01:48,005 JVMStabilityInspector.java:82 - Exiting
due to error while processing commit log during initialization.
org.apache.cassandra.io.FSWriteError: java.io.IOException: An attempt was made to move the
file pointer before the beginning of the file

Perhaps the index you are using is not big enough and it goes negative.

Is the relationship between max_mutation_size_in_kb and commitlog_segment_size_in_mb important
to preserve? In our limited stress test we are finding mutation size already over 512mb and
we expect more data in our sstables and associated materialized views.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message