cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From marc...@apache.org
Subject [2/2] cassandra git commit: Merge branch 'cassandra-2.1' into trunk
Date Tue, 03 Mar 2015 10:22:18 GMT
Merge branch 'cassandra-2.1' into trunk

Conflicts:
	src/java/org/apache/cassandra/db/compaction/CompactionManager.java
	src/java/org/apache/cassandra/service/ActiveRepairService.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2818ca4c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2818ca4c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2818ca4c

Branch: refs/heads/trunk
Commit: 2818ca4cf24f05a75041d220af4d3b0aa5203dbf
Parents: 7cc51f7 2f7077c
Author: Marcus Eriksson <marcuse@apache.org>
Authored: Tue Mar 3 10:44:21 2015 +0100
Committer: Marcus Eriksson <marcuse@apache.org>
Committed: Tue Mar 3 10:44:21 2015 +0100

----------------------------------------------------------------------
 CHANGES.txt                                     |  1 +
 .../apache/cassandra/db/ColumnFamilyStore.java  | 14 +++++++++++++
 .../db/compaction/CompactionManager.java        | 22 +++++++++++++-------
 .../cassandra/service/ActiveRepairService.java  |  9 --------
 4 files changed, 29 insertions(+), 17 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2818ca4c/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index 3892bbb,c3c7a19..23f9590
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,65 -1,5 +1,66 @@@
 +3.0
 + * Add role based access control (CASSANDRA-7653, 8650, 7216, 8760)
 + * Avoid accessing partitioner through StorageProxy (CASSANDRA-8244, 8268)
 + * Upgrade Metrics library and remove depricated metrics (CASSANDRA-5657)
 + * Serializing Row cache alternative, fully off heap (CASSANDRA-7438)
 + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707)
 + * Make CassandraException unchecked, extend RuntimeException (CASSANDRA-8560)
 + * Support direct buffer decompression for reads (CASSANDRA-8464)
 + * DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039)
 + * Group sstables for anticompaction correctly (CASSANDRA-8578)
 + * Add ReadFailureException to native protocol, respond
 +   immediately when replicas encounter errors while handling
 +   a read request (CASSANDRA-7886)
 + * Switch CommitLogSegment from RandomAccessFile to nio (CASSANDRA-8308)
 + * Allow mixing token and partition key restrictions (CASSANDRA-7016)
 + * Support index key/value entries on map collections (CASSANDRA-8473)
 + * Modernize schema tables (CASSANDRA-8261)
 + * Support for user-defined aggregation functions (CASSANDRA-8053)
 + * Fix NPE in SelectStatement with empty IN values (CASSANDRA-8419)
 + * Refactor SelectStatement, return IN results in natural order instead
 +   of IN value list order and ignore duplicate values in partition key IN restrictions (CASSANDRA-7981)
 + * Support UDTs, tuples, and collections in user-defined
 +   functions (CASSANDRA-7563)
 + * Fix aggregate fn results on empty selection, result column name,
 +   and cqlsh parsing (CASSANDRA-8229)
 + * Mark sstables as repaired after full repair (CASSANDRA-7586)
 + * Extend Descriptor to include a format value and refactor reader/writer
 +   APIs (CASSANDRA-7443)
 + * Integrate JMH for microbenchmarks (CASSANDRA-8151)
 + * Keep sstable levels when bootstrapping (CASSANDRA-7460)
 + * Add Sigar library and perform basic OS settings check on startup (CASSANDRA-7838)
 + * Support for aggregation functions (CASSANDRA-4914)
 + * Remove cassandra-cli (CASSANDRA-7920)
 + * Accept dollar quoted strings in CQL (CASSANDRA-7769)
 + * Make assassinate a first class command (CASSANDRA-7935)
 + * Support IN clause on any partition key column (CASSANDRA-7855)
 + * Support IN clause on any clustering column (CASSANDRA-4762)
 + * Improve compaction logging (CASSANDRA-7818)
 + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917)
 + * Do anticompaction in groups (CASSANDRA-6851)
 + * Support user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 7781, 7929,
 +   7924, 7812, 8063, 7813, 7708)
 + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416)
 + * Move sstable RandomAccessReader to nio2, which allows using the
 +   FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050)
 + * Remove CQL2 (CASSANDRA-5918)
 + * Add Thrift get_multi_slice call (CASSANDRA-6757)
 + * Optimize fetching multiple cells by name (CASSANDRA-6933)
 + * Allow compilation in java 8 (CASSANDRA-7028)
 + * Make incremental repair default (CASSANDRA-7250)
 + * Enable code coverage thru JaCoCo (CASSANDRA-7226)
 + * Switch external naming of 'column families' to 'tables' (CASSANDRA-4369) 
 + * Shorten SSTable path (CASSANDRA-6962)
 + * Use unsafe mutations for most unit tests (CASSANDRA-6969)
 + * Fix race condition during calculation of pending ranges (CASSANDRA-7390)
 + * Fail on very large batch sizes (CASSANDRA-8011)
 + * Improve concurrency of repair (CASSANDRA-6455, 8208)
 + * Select optimal CRC32 implementation at runtime (CASSANDRA-8614)
 + * Evaluate MurmurHash of Token once per query (CASSANDRA-7096)
 +
 +
  2.1.4
+  * Pick sstables for validation as late as possible inc repairs (CASSANDRA-8366)
   * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
   * Fix parallelism adjustment in range and secondary index queries
     when the first fetch does not satisfy the limit (CASSANDRA-8856)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2818ca4c/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
----------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2818ca4c/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 8b96c3a,e54a25f..66dd4b7
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@@ -985,29 -951,25 +985,35 @@@ public class CompactionManager implemen
              {
                  // flush first so everyone is validating data that is as similar as possible
                  StorageService.instance.forceKeyspaceFlush(cfs.keyspace.getName(), cfs.name);
 -                // we don't mark validating sstables as compacting in DataTracker, so we
have to mark them referenced
 -                // instead so they won't be cleaned up if they do get compacted during the
validation
 -                if (validator.desc.parentSessionId == null || ActiveRepairService.instance.getParentRepairSession(validator.desc.parentSessionId)
== null)
 -                    sstables = cfs.selectAndReference(ColumnFamilyStore.ALL_SSTABLES).refs;
 -                else
 -                {
 -                    ColumnFamilyStore.RefViewFragment refView = cfs.selectAndReference(ColumnFamilyStore.UNREPAIRED_SSTABLES);
 -                    sstables = refView.refs;
 -                    Set<SSTableReader> currentlyRepairing = ActiveRepairService.instance.currentlyRepairing(cfs.metadata.cfId,
validator.desc.parentSessionId);
 +                ActiveRepairService.ParentRepairSession prs = ActiveRepairService.instance.getParentRepairSession(validator.desc.parentSessionId);
++                ColumnFamilyStore.RefViewFragment sstableCandidates = cfs.selectAndReference(prs.isIncremental
? ColumnFamilyStore.UNREPAIRED_SSTABLES : ColumnFamilyStore.ALL_SSTABLES);
 +                Set<SSTableReader> sstablesToValidate = new HashSet<>();
-                 for (SSTableReader sstable : cfs.getSSTables())
+ 
 -                    if (!Sets.intersection(currentlyRepairing, Sets.newHashSet(refView.sstables)).isEmpty())
++                for (SSTableReader sstable : sstableCandidates.sstables)
 +                {
-                     if (new Bounds<>(sstable.first.getToken(), sstable.last.getToken()).intersects(Arrays.asList(validator.desc.range)))
++                    if (!(new Bounds<>(sstable.first.getToken(), sstable.last.getToken()).intersects(Arrays.asList(validator.desc.range))))
                      {
-                         if (!prs.isIncremental || !sstable.isRepaired())
-                         {
-                             sstablesToValidate.add(sstable);
-                         }
 -                        logger.error("Cannot start multiple repair sessions over the same
sstables");
 -                        throw new RuntimeException("Cannot start multiple repair sessions
over the same sstables");
++                        sstablesToValidate.add(sstable);
                      }
 +                }
+ 
 -                    ActiveRepairService.instance.getParentRepairSession(validator.desc.parentSessionId).addSSTables(cfs.metadata.cfId,
refView.sstables);
 +                Set<SSTableReader> currentlyRepairing = ActiveRepairService.instance.currentlyRepairing(cfs.metadata.cfId,
validator.desc.parentSessionId);
 +
 +                if (!Sets.intersection(currentlyRepairing, sstablesToValidate).isEmpty())
 +                {
 +                    logger.error("Cannot start multiple repair sessions over the same sstables");
 +                    throw new RuntimeException("Cannot start multiple repair sessions over
the same sstables");
                  }
-                 prs.addSSTables(cfs.metadata.cfId, sstablesToValidate);
  
-                 sstables = prs.getAndReferenceSSTablesInRange(cfs.metadata.cfId, validator.desc.range);
++                sstables = Refs.tryRef(sstablesToValidate);
++                if (sstables == null)
++                {
++                    logger.error("Could not reference sstables");
++                    throw new RuntimeException("Could not reference sstables");
++                }
++                sstableCandidates.release();
++                prs.addSSTables(cfs.metadata.cfId, sstablesToValidate);
 +
                  if (validator.gcBefore > 0)
                      gcBefore = validator.gcBefore;
                  else

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2818ca4c/src/java/org/apache/cassandra/service/ActiveRepairService.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/service/ActiveRepairService.java
index a9aed20,f71cb6b..faa32c2
--- a/src/java/org/apache/cassandra/service/ActiveRepairService.java
+++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java
@@@ -432,15 -430,15 +432,6 @@@ public class ActiveRepairServic
              return new Refs<>(references.build());
          }
  
-         public synchronized Refs<SSTableReader> getAndReferenceSSTablesInRange(UUID
cfId, Range<Token> range)
 -        public void addSSTables(UUID cfId, Collection<SSTableReader> sstables)
--        {
-             Refs<SSTableReader> sstables = getAndReferenceSSTables(cfId);
-             for (SSTableReader sstable : ImmutableList.copyOf(sstables))
-                 if (!new Bounds<>(sstable.first.getToken(), sstable.last.getToken()).intersects(Arrays.asList(range)))
-                     sstables.release(sstable);
-             return sstables;
 -            Set<SSTableReader> existingSSTables = this.sstableMap.get(cfId);
 -            if (existingSSTables == null)
 -                existingSSTables = new HashSet<>();
 -            existingSSTables.addAll(sstables);
 -            this.sstableMap.put(cfId, existingSSTables);
--        }
--
          @Override
          public String toString()
          {


Mime
View raw message