Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E35F1200C4B for ; Mon, 20 Mar 2017 20:25:33 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id E2517160B71; Mon, 20 Mar 2017 19:25:33 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 42D98160B8F for ; Mon, 20 Mar 2017 20:25:33 +0100 (CET) Received: (qmail 37067 invoked by uid 500); 20 Mar 2017 19:25:31 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 36581 invoked by uid 99); 20 Mar 2017 19:25:31 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 20 Mar 2017 19:25:31 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 6C35CE97E7; Mon, 20 Mar 2017 19:25:29 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: spod@apache.org To: commits@cassandra.apache.org Date: Mon, 20 Mar 2017 19:25:35 -0000 Message-Id: In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0 archived-at: Mon, 20 Mar 2017 19:25:34 -0000 Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f4ba9083 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f4ba9083 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f4ba9083 Branch: refs/heads/trunk Commit: f4ba9083e8251144d14cf367b8ef7d23c65ff2da Parents: 0eebc6e 06316df Author: Stefan Podkowinski Authored: Mon Mar 20 19:44:10 2017 +0100 Committer: Stefan Podkowinski Committed: Mon Mar 20 19:46:56 2017 +0100 ---------------------------------------------------------------------- CHANGES.txt | 1 + .../cassandra/db/compaction/CompactionManager.java | 12 +++++++++++- 2 files changed, 12 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4ba9083/CHANGES.txt ---------------------------------------------------------------------- diff --cc CHANGES.txt index 10402f3,27dd343..6021315 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,26 -1,8 +1,27 @@@ -2.2.10 +3.0.13 + * Fix CONTAINS filtering for null collections (CASSANDRA-13246) + * Applying: Use a unique metric reservoir per test run when using Cassandra-wide metrics residing in MBeans (CASSANDRA-13216) + * Propagate row deletions in 2i tables on upgrade (CASSANDRA-13320) + * Slice.isEmpty() returns false for some empty slices (CASSANDRA-13305) + * Add formatted row output to assertEmpty in CQL Tester (CASSANDRA-13238) +Merged from 2.2: + * Don't anti-compact repaired data to avoid inconsistencies (CASSANDRA-13153) * Wrong logger name in AnticompactionTask (CASSANDRA-13343) + * Commitlog replay may fail if last mutation is within 4 bytes of end of segment (CASSANDRA-13282) * Fix queries updating multiple time the same list (CASSANDRA-13130) * Fix GRANT/REVOKE when keyspace isn't specified (CASSANDRA-13053) + + +3.0.12 + * Prevent data loss on upgrade 2.1 - 3.0 by adding component separator to LogRecord absolute path (CASSANDRA-13294) + * Improve testing on macOS by eliminating sigar logging (CASSANDRA-13233) + * Cqlsh copy-from should error out when csv contains invalid data for collections (CASSANDRA-13071) + * Update c.yaml doc for offheap memtables (CASSANDRA-13179) + * Faster StreamingHistogram (CASSANDRA-13038) + * Legacy deserializer can create unexpected boundary range tombstones (CASSANDRA-13237) + * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070) + * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185) +Merged from 2.2: * Avoid race on receiver by starting streaming sender thread after sending init message (CASSANDRA-12886) * Fix "multiple versions of ant detected..." when running ant test (CASSANDRA-13232) * Coalescing strategy sleeps too much (CASSANDRA-13090) http://git-wip-us.apache.org/repos/asf/cassandra/blob/f4ba9083/src/java/org/apache/cassandra/db/compaction/CompactionManager.java ---------------------------------------------------------------------- diff --cc src/java/org/apache/cassandra/db/compaction/CompactionManager.java index 28140e0,d21f1e8..8d470d5 --- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java +++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java @@@ -22,7 -22,6 +22,8 @@@ import java.io.IOException import java.lang.management.ManagementFactory; import java.util.*; import java.util.concurrent.*; +import java.util.function.Predicate; ++import java.util.stream.Collectors; import javax.management.MBeanServer; import javax.management.ObjectName; import javax.management.openmbean.OpenDataException; @@@ -1222,7 -1212,21 +1223,16 @@@ public class CompactionManager implemen logger.info("Performing anticompaction on {} sstables", repaired.originals().size()); //Group SSTables - Collection> groupedSSTables = cfs.getCompactionStrategyManager().groupSSTablesForAntiCompaction(repaired.originals()); + Set sstables = repaired.originals(); + + // Repairs can take place on both unrepaired (incremental + full) and repaired (full) data. + // Although anti-compaction could work on repaired sstables as well and would result in having more accurate + // repairedAt values for these, we still avoid anti-compacting already repaired sstables, as we currently don't + // make use of any actual repairedAt value and splitting up sstables just for that is not worth it at this point. - Set unrepairedSSTables = ImmutableSet.copyOf(Iterables.filter(sstables, new Predicate() - { - public boolean apply(SSTableReader input) - { - return !input.isRepaired(); - } - })); ++ Set unrepairedSSTables = sstables.stream().filter((s) -> !s.isRepaired()).collect(Collectors.toSet()); ++ ++ Collection> groupedSSTables = cfs.getCompactionStrategyManager().groupSSTablesForAntiCompaction(unrepairedSSTables); + - Collection> groupedSSTables = cfs.getCompactionStrategy().groupSSTablesForAntiCompaction(unrepairedSSTables); // iterate over sstables to check if the repaired / unrepaired ranges intersect them. int antiCompactedSSTableCount = 0; for (Collection sstableGroup : groupedSSTables)