Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5726E10B99 for ; Wed, 5 Feb 2014 17:51:51 +0000 (UTC) Received: (qmail 85682 invoked by uid 500); 5 Feb 2014 17:51:50 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 85549 invoked by uid 500); 5 Feb 2014 17:51:49 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 85524 invoked by uid 99); 5 Feb 2014 17:51:48 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Feb 2014 17:51:48 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 7531591C5E9; Wed, 5 Feb 2014 17:51:48 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: slebresne@apache.org To: commits@cassandra.apache.org Date: Wed, 05 Feb 2014 17:51:49 -0000 Message-Id: <4969bbd109f948c2b7cb0734b42279b4@git.apache.org> In-Reply-To: <8acd9be512e64e60bd36ac88379392f6@git.apache.org> References: <8acd9be512e64e60bd36ac88379392f6@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [2/3] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0 Merge branch 'cassandra-1.2' into cassandra-2.0 Conflicts: CHANGES.txt src/java/org/apache/cassandra/db/AtomicSortedColumns.java src/java/org/apache/cassandra/db/DeletionInfo.java Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/58e94818 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/58e94818 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/58e94818 Branch: refs/heads/trunk Commit: 58e948185e214dbdc68e4ce533edb4dfa5430b51 Parents: 49bb972 adcb713 Author: Sylvain Lebresne Authored: Wed Feb 5 18:42:00 2014 +0100 Committer: Sylvain Lebresne Committed: Wed Feb 5 18:42:00 2014 +0100 ---------------------------------------------------------------------- CHANGES.txt | 5 +++++ .../org/apache/cassandra/db/AtomicSortedColumns.java | 7 ++++++- src/java/org/apache/cassandra/db/DeletionInfo.java | 14 ++++++++++++++ src/java/org/apache/cassandra/db/Memtable.java | 10 ++++++---- 4 files changed, 31 insertions(+), 5 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/58e94818/CHANGES.txt ---------------------------------------------------------------------- diff --cc CHANGES.txt index 9599e56,cfdd148..bba5f20 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,29 -1,21 +1,34 @@@ -1.2.16 ++2.0.6 ++Merged from 1.2: + * Fix partition and range deletes not triggering flush (CASSANDRA-6655) + -1.2.15 - * Move handling of migration event source to solve bootstrap race (CASSANDRA-6648) - * Make sure compaction throughput value doesn't overflow with int math (CASSANDRA-6647) - + -1.2.14 - * Reverted code to limit CQL prepared statement cache by size (CASSANDRA-6592) - * add cassandra.default_messaging_version property to allow easier - upgrading from 1.1 (CASSANDRA-6619) - * Allow executing CREATE statements multiple times (CASSANDRA-6471) - * Don't send confusing info with timeouts (CASSANDRA-6491) - * Don't resubmit counter mutation runnables internally (CASSANDRA-6427) - * Don't drop local mutations without a hint (CASSANDRA-6510) - * Don't allow null max_hint_window_in_ms (CASSANDRA-6419) - * Validate SliceRange start and finish lengths (CASSANDRA-6521) +2.0.5 + * Reduce garbage generated by bloom filter lookups (CASSANDRA-6609) + * Add ks.cf names to tombstone logging (CASSANDRA-6597) + * Use LOCAL_QUORUM for LWT operations at LOCAL_SERIAL (CASSANDRA-6495) + * Wait for gossip to settle before accepting client connections (CASSANDRA-4288) + * Delete unfinished compaction incrementally (CASSANDRA-6086) + * Allow specifying custom secondary index options in CQL3 (CASSANDRA-6480) + * Improve replica pinning for cache efficiency in DES (CASSANDRA-6485) + * Fix LOCAL_SERIAL from thrift (CASSANDRA-6584) + * Don't special case received counts in CAS timeout exceptions (CASSANDRA-6595) + * Add support for 2.1 global counter shards (CASSANDRA-6505) + * Fix NPE when streaming connection is not yet established (CASSANDRA-6210) + * Avoid rare duplicate read repair triggering (CASSANDRA-6606) + * Fix paging discardFirst (CASSANDRA-6555) + * Fix ArrayIndexOutOfBoundsException in 2ndary index query (CASSANDRA-6470) + * Release sstables upon rebuilding 2i (CASSANDRA-6635) + * Add AbstractCompactionStrategy.startup() method (CASSANDRA-6637) + * SSTableScanner may skip rows during cleanup (CASSANDRA-6638) + * sstables from stalled repair sessions can resurrect deleted data (CASSANDRA-6503) + * Switch stress to use ITransportFactory (CASSANDRA-6641) + * Fix IllegalArgumentException during prepare (CASSANDRA-6592) + * Fix possible loss of 2ndary index entries during compaction (CASSANDRA-6517) + * Fix direct Memory on architectures that do not support unaligned long access + (CASSANDRA-6628) + * Let scrub optionally skip broken counter partitions (CASSANDRA-5930) +Merged from 1.2: * fsync compression metadata (CASSANDRA-6531) * Validate CF existence on execution for prepared statement (CASSANDRA-6535) * Add ability to throttle batchlog replay (CASSANDRA-6550) http://git-wip-us.apache.org/repos/asf/cassandra/blob/58e94818/src/java/org/apache/cassandra/db/AtomicSortedColumns.java ---------------------------------------------------------------------- diff --cc src/java/org/apache/cassandra/db/AtomicSortedColumns.java index 1c0bf1b,d6c861b..d3a979c --- a/src/java/org/apache/cassandra/db/AtomicSortedColumns.java +++ b/src/java/org/apache/cassandra/db/AtomicSortedColumns.java @@@ -178,19 -194,15 +178,24 @@@ public class AtomicSortedColumns extend { sizeDelta = 0; current = ref.get(); - DeletionInfo newDelInfo = current.deletionInfo.copy().add(cm.deletionInfo()); + DeletionInfo newDelInfo = current.deletionInfo; - if (cm.getDeletionInfo().mayModify(newDelInfo)) ++ if (cm.deletionInfo().mayModify(newDelInfo)) + { - newDelInfo = current.deletionInfo.copy().add(cm.getDeletionInfo()); ++ newDelInfo = current.deletionInfo.copy().add(cm.deletionInfo()); + sizeDelta += newDelInfo.dataSize() - current.deletionInfo.dataSize(); + } modified = new Holder(current.map.clone(), newDelInfo); - for (IColumn column : cm.getSortedColumns()) + if (cm.deletionInfo().hasRanges()) + { + for (Column currentColumn : Iterables.concat(current.map.values(), cm)) + { + if (cm.deletionInfo().isDeleted(currentColumn)) + indexer.remove(currentColumn); + } + } + + for (Column column : cm) { sizeDelta += modified.addColumn(transformation.apply(column), allocator, indexer); // bail early if we know we've been beaten http://git-wip-us.apache.org/repos/asf/cassandra/blob/58e94818/src/java/org/apache/cassandra/db/DeletionInfo.java ---------------------------------------------------------------------- diff --cc src/java/org/apache/cassandra/db/DeletionInfo.java index 13fc824,91af9fd..23f46bf --- a/src/java/org/apache/cassandra/db/DeletionInfo.java +++ b/src/java/org/apache/cassandra/db/DeletionInfo.java @@@ -266,11 -216,20 +266,25 @@@ public class DeletionInf return size + (ranges == null ? 0 : ranges.dataSize()); } + public boolean hasRanges() + { + return ranges != null && !ranges.isEmpty(); + } + + public int rangeCount() + { - return ranges == null ? 0 : ranges.size(); ++ return hasRanges() ? ranges.size() : 0; + } + + /** + * Whether this deletion info may modify the provided one if added to it. + */ + public boolean mayModify(DeletionInfo delInfo) + { + return topLevel.markedForDeleteAt > delInfo.topLevel.markedForDeleteAt - || ranges == null; ++ || hasRanges(); + } + @Override public String toString() { http://git-wip-us.apache.org/repos/asf/cassandra/blob/58e94818/src/java/org/apache/cassandra/db/Memtable.java ---------------------------------------------------------------------- diff --cc src/java/org/apache/cassandra/db/Memtable.java index 1ca7e39,b229060..01170d6 --- a/src/java/org/apache/cassandra/db/Memtable.java +++ b/src/java/org/apache/cassandra/db/Memtable.java @@@ -180,22 -190,25 +180,24 @@@ public class Memtabl private void resolve(DecoratedKey key, ColumnFamily cf, SecondaryIndexManager.Updater indexer) { - ColumnFamily previous = columnFamilies.get(key); + AtomicSortedColumns previous = rows.get(key); + long sizeDelta = 0; if (previous == null) { - // AtomicSortedColumns doesn't work for super columns (see #3821) - ColumnFamily empty = cf.cloneMeShallow(cf.isSuper() ? ThreadSafeSortedColumns.factory() : AtomicSortedColumns.factory(), false); + AtomicSortedColumns empty = cf.cloneMeShallow(AtomicSortedColumns.factory, false); // We'll add the columns later. This avoids wasting works if we get beaten in the putIfAbsent - previous = columnFamilies.putIfAbsent(new DecoratedKey(key.token, allocator.clone(key.key)), empty); + previous = rows.putIfAbsent(new DecoratedKey(key.token, allocator.clone(key.key)), empty); if (previous == null) + { previous = empty; + sizeDelta += empty.deletionInfo().dataSize(); + } } - long sizeDelta = previous.addAllWithSizeDelta(cf, allocator, localCopyFunction, indexer); + sizeDelta = previous.addAllWithSizeDelta(cf, allocator, localCopyFunction, indexer); currentSize.addAndGet(sizeDelta); - currentOperations.addAndGet((cf.getColumnCount() == 0) - ? cf.isMarkedForDelete() ? 1 : 0 - : cf.getColumnCount()); + currentOperations.addAndGet(cf.getColumnCount() + (cf.isMarkedForDelete() ? 1 : 0) + cf.deletionInfo().rangeCount()); } // for debugging