cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tania S Engel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-13565) Materialized view usage of commit logs requires large mutation but commitlog_segment_size_in_mb=2048 causes exception
Date Tue, 27 Jun 2017 15:09:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-13565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064975#comment-16064975
] 

Tania S Engel commented on CASSANDRA-13565:
-------------------------------------------

Fabulous. Thanks for the explanation, I understand enough to fix it. We actually switched
away from using MVs from these particular tables due to their heavy inserts, and the fact
we kept running into memory issues when joining.

How about my other question ...setting  commitlog_segment_size_in_mb=2048 we get :
ERROR [COMMIT-LOG-ALLOCATOR] 2017-05-31 17:01:48,005 JVMStabilityInspector.java:82 - Exiting
due to error while processing commit log during initialization.
org.apache.cassandra.io.FSWriteError: java.io.IOException: An attempt was made to move the
file pointer before the beginning of the file


> Materialized view usage of commit logs requires large mutation but commitlog_segment_size_in_mb=2048
causes exception
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-13565
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13565
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Configuration, Materialized Views, Streaming and Messaging
>         Environment: Cassandra 3.9.0, Windows 
>            Reporter: Tania S Engel
>         Attachments: CQLforTable.png
>
>
> We will be upgrading to 3.10 for CASSANDRA-11670. However, there is another scenario
(not applyunsafe during JOIN) which leads to :
> 	java.lang.IllegalArgumentException: Mutation of 525.847MiB is too large for the maximum
size of 512.000MiB
> 	            at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:262)
~[apache-cassandra-3.9.0.jar:3.9.0]
> 	            at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:493)
~[apache-cassandra-3.9.0.jar:3.9.0]
> 	            at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:396)
~[apache-cassandra-3.9.0.jar:3.9.0]
> 	            at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:215)
~[apache-cassandra-3.9.0.jar:3.9.0]
> 	            at org.apache.cassandra.db.Mutation.apply(Mutation.java:227)
~[apache-cassandra-3.9.0.jar:3.9.0]
> 	            at org.apache.cassandra.batchlog.BatchlogManager.store(BatchlogManager.java:147)
~[apache-cassandra-3.9.0.jar:3.9.0]
> 	            at org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:797)
~[apache-cassandra-3.9.0.jar:3.9.0]
> 	            at org.apache.cassandra.db.view.ViewBuilder.buildKey(ViewBuilder.java:96)
~[apache-cassandra-3.9.0.jar:3.9.0]
> 	            at org.apache.cassandra.db.view.ViewBuilder.run(ViewBuilder.java:165)
~[apache-cassandra-3.9.0.jar:3.9.0]
> 	            at org.apache.cassandra.db.compaction.CompactionManager$14.run(CompactionManager.java:1591)
[apache-cassandra-3.9.0.jar:3.9.0]
> 	            at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_66]
> 	            at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_66]
> 	            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_66]
> 	            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_66]
> 	            at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66] 
> Due to the relationship of max_mutation_size_in_kb and commitlog_segment_size_in_mb,
we increased commitlog_segment_size_in_mb and left Cassandra to calculate max_mutation_size_in_kb
as half the size commitlog_segment_size_in_mb * 1024.
>  However, we have found that if we set commitlog_segment_size_in_mb=2048 we get an exception
upon starting Cassandra, when it is creating a new commit log.
> ERROR [COMMIT-LOG-ALLOCATOR] 2017-05-31 17:01:48,005 JVMStabilityInspector.java:82 -
Exiting due to error while processing commit log during initialization.
> org.apache.cassandra.io.FSWriteError: java.io.IOException: An attempt was made to move
the file pointer before the beginning of the file
> Perhaps the index you are using is not big enough and it goes negative.
> Is the relationship between max_mutation_size_in_kb and commitlog_segment_size_in_mb
important to preserve? In our limited stress test we are finding mutation size already over
512mb and we expect more data in our sstables and associated materialized views.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message