Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B72E4200BFE for ; Mon, 16 Jan 2017 10:19:33 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id B5E3D160B30; Mon, 16 Jan 2017 09:19:33 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id DD5BD160B22 for ; Mon, 16 Jan 2017 10:19:32 +0100 (CET) Received: (qmail 85249 invoked by uid 500); 16 Jan 2017 09:19:31 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 85238 invoked by uid 99); 16 Jan 2017 09:19:31 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 Jan 2017 09:19:31 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 732FE1A02B7 for ; Mon, 16 Jan 2017 09:19:31 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.199 X-Spam-Level: X-Spam-Status: No, score=-1.199 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id q0kJyERs4AYZ for ; Mon, 16 Jan 2017 09:19:30 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 9DDFF60DE8 for ; Mon, 16 Jan 2017 09:19:29 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id ADE0CE867C for ; Mon, 16 Jan 2017 09:19:28 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id A9A642528F for ; Mon, 16 Jan 2017 09:19:26 +0000 (UTC) Date: Mon, 16 Jan 2017 09:19:26 +0000 (UTC) From: "Sylvain Lebresne (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (CASSANDRA-12203) AssertionError on compaction after upgrade (2.1.9 -> 3.7) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 16 Jan 2017 09:19:33 -0000 [ https://issues.apache.org/jira/browse/CASSANDRA-12203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-12203: ----------------------------------------- Status: Ready to Commit (was: Patch Available) > AssertionError on compaction after upgrade (2.1.9 -> 3.7) > --------------------------------------------------------- > > Key: CASSANDRA-12203 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12203 > Project: Cassandra > Issue Type: Bug > Components: Core > Environment: Cassandra 3.7 (upgrade from 2.1.9) > Java version "1.8.0_91" > Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-83-generic x86_64) > Reporter: Roman S. Borschel > Assignee: Yuki Morishita > Priority: Critical > Fix For: 3.0.x, 3.x > > > After upgrading a Cassandra cluster from 2.1.9 to 3.7, one column family (using SizeTieredCompaction) repeatedly and continuously failed compaction (and thus also repair) across the cluster, with all nodes producing the following errors in the logs: > {noformat} > 016-07-14T09:29:47.96855 |srv=cassandra|ERROR: Exception in thread Thread[CompactionExecutor:3,1,main] > 2016-07-14T09:29:47.96858 |srv=cassandra|java.lang.AssertionError: null > 2016-07-14T09:29:47.96859 |srv=cassandra| at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$TombstoneTracker.openNew(UnfilteredDeserializer.java:650) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96860 |srv=cassandra| at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer$UnfilteredIterator.hasNext(UnfilteredDeserializer.java:423) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96860 |srv=cassandra| at org.apache.cassandra.db.UnfilteredDeserializer$OldFormatDeserializer.hasNext(UnfilteredDeserializer.java:298) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96860 |srv=cassandra| at org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.readStaticRow(SSTableSimpleIterator.java:133) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96861 |srv=cassandra| at org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:57) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96861 |srv=cassandra| at org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator$1.initializeIterator(BigTableScanner.java:334) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96862 |srv=cassandra| at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96862 |srv=cassandra| at org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.isReverseOrder(LazilyInitializedUnfilteredRowIterator.java:70) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96863 |srv=cassandra| at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:109) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96863 |srv=cassandra| at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:100) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96864 |srv=cassandra| at org.apache.cassandra.utils.MergeIterator$Candidate.consume(MergeIterator.java:408) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96864 |srv=cassandra| at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:203) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96865 |srv=cassandra| at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:156) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96865 |srv=cassandra| at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96866 |srv=cassandra| at org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:150) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96866 |srv=cassandra| at org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:72) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96867 |srv=cassandra| at org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:226) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96867 |srv=cassandra| at org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96867 |srv=cassandra| at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96868 |srv=cassandra| at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96868 |srv=cassandra| at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96869 |srv=cassandra| at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264) ~[apache-cassandra-3.7.jar:3.7] > 2016-07-14T09:29:47.96870 |srv=cassandra| at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_91] > 2016-07-14T09:29:47.96870 |srv=cassandra| at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_91] > 2016-07-14T09:29:47.96870 |srv=cassandra| at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_91] > 2016-07-14T09:29:47.96871 |srv=cassandra| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91] > 2016-07-14T09:29:47.96871 |srv=cassandra| at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] > {noformat} > Only 1 out of 14 tables was affected by this problem after the upgrade. > The schema of the affected table looks like this: > {noformat} > create columnfamily if not exists cassandra12203 > ( field1 uuid > , field2 ascii > , field3 int > , field4 text > , primary key (field1, field2, field3) > ); > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)