Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 36195200CB4 for ; Tue, 27 Jun 2017 17:09:17 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 34D7B160BDC; Tue, 27 Jun 2017 15:09:17 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 54BE5160BC6 for ; Tue, 27 Jun 2017 17:09:16 +0200 (CEST) Received: (qmail 69289 invoked by uid 500); 27 Jun 2017 15:09:13 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 69278 invoked by uid 99); 27 Jun 2017 15:09:13 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 27 Jun 2017 15:09:13 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 22E78C047B for ; Tue, 27 Jun 2017 15:09:13 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -99.202 X-Spam-Level: X-Spam-Status: No, score=-99.202 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id h-aEiQnP6GQZ for ; Tue, 27 Jun 2017 15:09:11 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 422EA5F60D for ; Tue, 27 Jun 2017 15:09:11 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 6EE7FE0069 for ; Tue, 27 Jun 2017 15:09:05 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 24BF824120 for ; Tue, 27 Jun 2017 15:09:00 +0000 (UTC) Date: Tue, 27 Jun 2017 15:09:00 +0000 (UTC) From: "Tania S Engel (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CASSANDRA-13565) Materialized view usage of commit logs requires large mutation but commitlog_segment_size_in_mb=2048 causes exception MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 27 Jun 2017 15:09:17 -0000 [ https://issues.apache.org/jira/browse/CASSANDRA-13565?page=3Dcom.atla= ssian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId= =3D16064975#comment-16064975 ]=20 Tania S Engel commented on CASSANDRA-13565: ------------------------------------------- Fabulous. Thanks for the explanation, I understand enough to fix it. We act= ually switched away from using MVs from these particular tables due to thei= r heavy inserts, and the fact we kept running into memory issues when joini= ng. How about my other question ...setting commitlog_segment_size_in_mb=3D2048= we get : ERROR [COMMIT-LOG-ALLOCATOR] 2017-05-31 17:01:48,005 JVMStabilityInspector.= java:82 - Exiting due to error while processing commit log during initializ= ation. org.apache.cassandra.io.FSWriteError: java.io.IOException: An attempt was m= ade to move the file pointer before the beginning of the file > Materialized view usage of commit logs requires large mutation but commit= log_segment_size_in_mb=3D2048 causes exception > -------------------------------------------------------------------------= -------------------------------------------- > > Key: CASSANDRA-13565 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1356= 5 > Project: Cassandra > Issue Type: Bug > Components: Configuration, Materialized Views, Streaming and Mes= saging > Environment: Cassandra 3.9.0, Windows=20 > Reporter: Tania S Engel > Attachments: CQLforTable.png > > > We will be upgrading to 3.10 for CASSANDRA-11670. However, there is anoth= er scenario (not applyunsafe during JOIN) which leads to : > =09java.lang.IllegalArgumentException: Mutation of 525.847MiB is too larg= e for the maximum size of 512.000MiB > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:262) ~[apach= e-cassandra-3.9.0.jar:3.9.0] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = org.apache.cassandra.db.Keyspace.apply(Keyspace.java:493) ~[apache-cassandr= a-3.9.0.jar:3.9.0] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = org.apache.cassandra.db.Keyspace.apply(Keyspace.java:396) ~[apache-cassandr= a-3.9.0.jar:3.9.0] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:215) ~[apache-ca= ssandra-3.9.0.jar:3.9.0] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = org.apache.cassandra.db.Mutation.apply(Mutation.java:227) ~[apache-cassandr= a-3.9.0.jar:3.9.0] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = org.apache.cassandra.batchlog.BatchlogManager.store(BatchlogManager.java:14= 7) ~[apache-cassandra-3.9.0.jar:3.9.0] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:797) ~= [apache-cassandra-3.9.0.jar:3.9.0] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = org.apache.cassandra.db.view.ViewBuilder.buildKey(ViewBuilder.java:96) ~[ap= ache-cassandra-3.9.0.jar:3.9.0] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = org.apache.cassandra.db.view.ViewBuilder.run(ViewBuilder.java:165) ~[apache= -cassandra-3.9.0.jar:3.9.0] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = org.apache.cassandra.db.compaction.CompactionManager$14.run(CompactionManag= er.java:1591) [apache-cassandra-3.9.0.jar:3.9.0] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na= :1.8.0_66] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_66] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1= 142) [na:1.8.0_66] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:= 617) [na:1.8.0_66] > =09=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 at = java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]=20 > Due to the relationship of max_mutation_size_in_kb and commitlog_segment_= size_in_mb, we increased commitlog_segment_size_in_mb and left Cassandra to= calculate max_mutation_size_in_kb as half the size commitlog_segment_size_= in_mb * 1024. > However, we have found that if we set commitlog_segment_size_in_mb=3D204= 8 we get an exception upon starting Cassandra, when it is creating a new co= mmit log. > ERROR [COMMIT-LOG-ALLOCATOR] 2017-05-31 17:01:48,005 JVMStabilityInspecto= r.java:82 - Exiting due to error while processing commit log during initial= ization. > org.apache.cassandra.io.FSWriteError: java.io.IOException: An attempt was= made to move the file pointer before the beginning of the file > Perhaps the index you are using is not big enough and it goes negative. > Is the relationship between max_mutation_size_in_kb and commitlog_segment= _size_in_mb important to preserve? In our limited stress test we are findin= g mutation size already over 512mb and we expect more data in our sstables = and associated materialized views. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org For additional commands, e-mail: commits-help@cassandra.apache.org