Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CAB2718E55 for ; Mon, 17 Aug 2015 06:56:36 +0000 (UTC) Received: (qmail 98411 invoked by uid 500); 17 Aug 2015 06:56:36 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 98269 invoked by uid 500); 17 Aug 2015 06:56:36 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 98153 invoked by uid 99); 17 Aug 2015 06:56:36 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 17 Aug 2015 06:56:36 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 442DFE00D6; Mon, 17 Aug 2015 06:56:36 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: marcuse@apache.org To: commits@cassandra.apache.org Date: Mon, 17 Aug 2015 06:56:38 -0000 Message-Id: <071013eaecb540769506e4e0480b8ad9@git.apache.org> In-Reply-To: <01db48fbd4a448ac8e4ecd35837da5c2@git.apache.org> References: <01db48fbd4a448ac8e4ecd35837da5c2@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [3/5] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2 Merge branch 'cassandra-2.1' into cassandra-2.2 Conflicts: src/java/org/apache/cassandra/db/ColumnFamilyStore.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/31db093c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/31db093c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/31db093c Branch: refs/heads/trunk Commit: 31db093c478f29f000481e1368b3b2addff33eb2 Parents: 0f287c4 26ff150 Author: Marcus Eriksson Authored: Mon Aug 17 08:34:24 2015 +0200 Committer: Marcus Eriksson Committed: Mon Aug 17 08:34:24 2015 +0200 ---------------------------------------------------------------------- CHANGES.txt | 1 + .../apache/cassandra/db/ColumnFamilyStore.java | 55 +++++++++++++++++--- .../DateTieredCompactionStrategy.java | 13 ++++- .../DateTieredCompactionStrategyTest.java | 1 + 4 files changed, 61 insertions(+), 9 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/31db093c/CHANGES.txt ---------------------------------------------------------------------- diff --cc CHANGES.txt index 6bc671d,4626899..1299aa2 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -17,23 -9,8 +17,24 @@@ Merged from 2.1 * Release snapshot selfRef when doing snapshot repair (CASSANDRA-9998) * Cannot replace token does not exist - DN node removed as Fat Client (CASSANDRA-9871) * Fix handling of enable/disable autocompaction (CASSANDRA-9899) - * Commit log segment recycling is disabled by default (CASSANDRA-9896) * Add consistency level to tracing ouput (CASSANDRA-9827) + * Remove repair snapshot leftover on startup (CASSANDRA-7357) + * Use random nodes for batch log when only 2 racks (CASSANDRA-8735) + * Ensure atomicity inside thrift and stream session (CASSANDRA-7757) + * Fix nodetool info error when the node is not joined (CASSANDRA-9031) +Merged from 2.0: ++ * Make getFullyExpiredSSTables less expensive (CASSANDRA-9882) + * Log when messages are dropped due to cross_node_timeout (CASSANDRA-9793) + * Don't track hotness when opening from snapshot for validation (CASSANDRA-9382) + + +2.2.0 + * Allow the selection of columns together with aggregates (CASSANDRA-9767) + * Fix cqlsh copy methods and other windows specific issues (CASSANDRA-9795) + * Don't wrap byte arrays in SequentialWriter (CASSANDRA-9797) + * sum() and avg() functions missing for smallint and tinyint types (CASSANDRA-9671) + * Revert CASSANDRA-9542 (allow native functions in UDA) (CASSANDRA-9771) +Merged from 2.1: * Fix MarshalException when upgrading superColumn family (CASSANDRA-9582) * Fix broken logging for "empty" flushes in Memtable (CASSANDRA-9837) * Handle corrupt files on startup (CASSANDRA-9686) http://git-wip-us.apache.org/repos/asf/cassandra/blob/31db093c/src/java/org/apache/cassandra/db/ColumnFamilyStore.java ---------------------------------------------------------------------- diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 8bda6b2,25b3e57..343ecee --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@@ -1389,17 -1370,56 +1389,58 @@@ public class ColumnFamilyStore implemen if (!sstables.iterator().hasNext()) return ImmutableSet.of(); - SSTableIntervalTree tree = data.getView().intervalTree; + - Set results = null; - for (SSTableReader sstable : sstables) ++ + List sortedByFirst = Lists.newArrayList(sstables); + Collections.sort(sortedByFirst, new Comparator() + { + @Override + public int compare(SSTableReader o1, SSTableReader o2) + { + return o1.first.compareTo(o2.first); + } + }); + List> intervals = new ArrayList<>(); + DecoratedKey first = null, last = null; + /* + normalize the intervals covered by the sstables + assume we have sstables like this (brackets representing first/last key in the sstable); + [ ] [ ] [ ] [ ] + [ ] [ ] + then we can, instead of searching the interval tree 6 times, normalize the intervals and + only query the tree 2 times, for these intervals; + [ ] [ ] + */ + for (SSTableReader sstable : sortedByFirst) { - Set overlaps = ImmutableSet.copyOf(tree.search(Interval.create(sstable.first, sstable.last))); - results = results == null ? overlaps : Sets.union(results, overlaps).immutableCopy(); + if (first == null) + { + first = sstable.first; + last = sstable.last; + } + else + { + if (sstable.first.compareTo(last) <= 0) // we do overlap + { + if (sstable.last.compareTo(last) > 0) + last = sstable.last; + } + else + { + intervals.add(Interval.create(first, last)); + first = sstable.first; + last = sstable.last; + } + } } - results = Sets.difference(results, ImmutableSet.copyOf(sstables)); + intervals.add(Interval.create(first, last)); - DataTracker.SSTableIntervalTree tree = data.getView().intervalTree; ++ SSTableIntervalTree tree = data.getView().intervalTree; + Set results = new HashSet<>(); + + for (Interval interval : intervals) + results.addAll(tree.search(interval)); - return results; + return Sets.difference(results, ImmutableSet.copyOf(sstables)); } /** http://git-wip-us.apache.org/repos/asf/cassandra/blob/31db093c/src/java/org/apache/cassandra/db/compaction/DateTieredCompactionStrategy.java ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/31db093c/test/unit/org/apache/cassandra/db/compaction/DateTieredCompactionStrategyTest.java ----------------------------------------------------------------------