cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From slebre...@apache.org
Subject [1/2] git commit: Merge branch 'cassandra-1.0' into cassandra-1.1
Date Fri, 27 Jul 2012 08:14:46 GMT
Updated Branches:
  refs/heads/cassandra-1.1 070c1d8dd -> ccd436c0f


Merge branch 'cassandra-1.0' into cassandra-1.1

Conflicts:
	CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ccd436c0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ccd436c0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ccd436c0

Branch: refs/heads/cassandra-1.1
Commit: ccd436c0fe0c75db09dd7cd87a516f005f4e7289
Parents: 070c1d8 7ff8e3c
Author: Sylvain Lebresne <sylvain@datastax.com>
Authored: Fri Jul 27 10:14:34 2012 +0200
Committer: Sylvain Lebresne <sylvain@datastax.com>
Committed: Fri Jul 27 10:14:34 2012 +0200

----------------------------------------------------------------------
 CHANGES.txt                                        |    2 ++
 .../cassandra/db/compaction/CompactionManager.java |    9 ++++++---
 2 files changed, 8 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ccd436c0/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index bcf56b7,b3fa1a8..fd6991c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,155 -1,6 +1,157 @@@
 -1.0.11
 +1.1.3
 + * (JMX) rename getRangeKeySample to sampleKeyRange to avoid returning
 +   multi-MB results as an attribute (CASSANDRA-4452)
 + * flush based on data size, not throughput; overwritten columns no 
 +   longer artificially inflate liveRatio (CASSANDRA-4399)
 + * update default commitlog segment size to 32MB and total commitlog
 +   size to 32/1024 MB for 32/64 bit JVMs, respectively (CASSANDRA-4422)
 + * avoid using global partitioner to estimate ranges in index sstables
 +   (CASSANDRA-4403)
 + * restore pre-CASSANDRA-3862 approach to removing expired tombstones
 +   from row cache during compaction (CASSANDRA-4364)
 + * (stress) support for CQL prepared statements (CASSANDRA-3633)
 + * Correctly catch exception when Snappy cannot be loaded (CASSANDRA-4400)
 + * (cql3) Support ORDER BY when IN condition is given in WHERE clause (CASSANDRA-4327)
 + * (cql3) delete "component_index" column on DROP TABLE call (CASSANDRA-4420)
 + * change nanoTime() to currentTimeInMillis() in schema related code (CASSANDRA-4432)
 + * add a token generation tool (CASSANDRA-3709)
 + * Fix LCS bug with sstable containing only 1 row (CASSANDRA-4411)
 + * fix "Can't Modify Index Name" problem on CF update (CASSANDRA-4439)
 + * Fix assertion error in getOverlappingSSTables during repair (CASSANDRA-4456)
 + * fix nodetool's setcompactionthreshold command (CASSANDRA-4455)
 + * Ensure compacted files are never used, to avoid counter overcount (CASSANDRA-4436)
 +Merged from 1.0:
 + * Push the validation of secondary index values to the SecondaryIndexManager (CASSANDRA-4240)
   * allow dropping columns shadowed by not-yet-expired supercolumn or row
     tombstones in PrecompactedRow (CASSANDRA-4396)
 + * fix 1.0.x node join to mixed version cluster, other nodes >= 1.1 (CASSANDRA-4195)
 + * Fix LCS splitting sstable base on uncompressed size (CASSANDRA-4419)
 + * Bootstraps that fail are detected upon restart and will retry safely without
 +   needing to delete existing data first (CASSANDRA-4427)
 + * (cqlsh) add a COPY TO command to copy a CF to a CSV file (CASSANDRA-4434)
++ * Don't purge columns during upgradesstables (CASSANDRA-4462)
++ * Push the validation of secondary index values to the SecondaryIndexManager (CASSANDRA-4240)
 +
 +
 +1.1.2
 + * Fix cleanup not deleting index entries (CASSANDRA-4379)
 + * Use correct partitioner when saving + loading caches (CASSANDRA-4331)
 + * Check schema before trying to export sstable (CASSANDRA-2760)
 + * Raise a meaningful exception instead of NPE when PFS encounters
 +   an unconfigured node + no default (CASSANDRA-4349)
 + * fix bug in sstable blacklisting with LCS (CASSANDRA-4343)
 + * LCS no longer promotes tiny sstables out of L0 (CASSANDRA-4341)
 + * skip tombstones during hint replay (CASSANDRA-4320)
 + * fix NPE in compactionstats (CASSANDRA-4318)
 + * enforce 1m min keycache for auto (CASSANDRA-4306)
 + * Have DeletedColumn.isMFD always return true (CASSANDRA-4307)
 + * (cql3) exeption message for ORDER BY constraints said primary filter can be
 +    an IN clause, which is misleading (CASSANDRA-4319)
 + * (cql3) Reject (not yet supported) creation of 2ndardy indexes on tables with
 +   composite primary keys (CASSANDRA-4328)
 + * Set JVM stack size to 160k for java 7 (CASSANDRA-4275)
 + * cqlsh: add COPY command to load data from CSV flat files (CASSANDRA-4012)
 + * CFMetaData.fromThrift to throw ConfigurationException upon error (CASSANDRA-4353)
 + * Use CF comparator to sort indexed columns in SecondaryIndexManager
 +   (CASSANDRA-4365)
 + * add strategy_options to the KSMetaData.toString() output (CASSANDRA-4248)
 + * (cql3) fix range queries containing unqueried results (CASSANDRA-4372)
 + * (cql3) allow updating column_alias types (CASSANDRA-4041)
 + * (cql3) Fix deletion bug (CASSANDRA-4193)
 + * Fix computation of overlapping sstable for leveled compaction (CASSANDRA-4321)
 + * Improve scrub and allow to run it offline (CASSANDRA-4321)
 + * Fix assertionError in StorageService.bulkLoad (CASSANDRA-4368)
 + * (cqlsh) add option to authenticate to a keyspace at startup (CASSANDRA-4108)
 + * (cqlsh) fix ASSUME functionality (CASSANDRA-4352)
 + * Fix ColumnFamilyRecordReader to not return progress > 100% (CASSANDRA-3942)
 +Merged from 1.0:
 + * Set gc_grace on index CF to 0 (CASSANDRA-4314)
 +
 +
 +1.1.1
 + * allow larger cache capacities than 2GB (CASSANDRA-4150)
 + * add getsstables command to nodetool (CASSANDRA-4199)
 + * apply parent CF compaction settings to secondary index CFs (CASSANDRA-4280)
 + * preserve commitlog size cap when recycling segments at startup
 +   (CASSANDRA-4201)
 + * (Hadoop) fix split generation regression (CASSANDRA-4259)
 + * ignore min/max compactions settings in LCS, while preserving
 +   behavior that min=max=0 disables autocompaction (CASSANDRA-4233)
 + * log number of rows read from saved cache (CASSANDRA-4249)
 + * calculate exact size required for cleanup operations (CASSANDRA-1404)
 + * avoid blocking additional writes during flush when the commitlog
 +   gets behind temporarily (CASSANDRA-1991)
 + * enable caching on index CFs based on data CF cache setting (CASSANDRA-4197)
 + * warn on invalid replication strategy creation options (CASSANDRA-4046)
 + * remove [Freeable]Memory finalizers (CASSANDRA-4222)
 + * include tombstone size in ColumnFamily.size, which can prevent OOM
 +   during sudden mass delete operations by yielding a nonzero liveRatio
 +   (CASSANDRA-3741)
 + * Open 1 sstableScanner per level for leveled compaction (CASSANDRA-4142)
 + * Optimize reads when row deletion timestamps allow us to restrict
 +   the set of sstables we check (CASSANDRA-4116)
 + * add support for commitlog archiving and point-in-time recovery
 +   (CASSANDRA-3690)
 + * avoid generating redundant compaction tasks during streaming
 +   (CASSANDRA-4174)
 + * add -cf option to nodetool snapshot, and takeColumnFamilySnapshot to
 +   StorageService mbean (CASSANDRA-556)
 + * optimize cleanup to drop entire sstables where possible (CASSANDRA-4079)
 + * optimize truncate when autosnapshot is disabled (CASSANDRA-4153)
 + * update caches to use byte[] keys to reduce memory overhead (CASSANDRA-3966)
 + * add column limit to cli (CASSANDRA-3012, 4098)
 + * clean up and optimize DataOutputBuffer, used by CQL compression and
 +   CompositeType (CASSANDRA-4072)
 + * optimize commitlog checksumming (CASSANDRA-3610)
 + * identify and blacklist corrupted SSTables from future compactions 
 +   (CASSANDRA-2261)
 + * Move CfDef and KsDef validation out of thrift (CASSANDRA-4037)
 + * Expose API to repair a user provided range (CASSANDRA-3912)
 + * Add way to force the cassandra-cli to refresh its schema (CASSANDRA-4052)
 + * Avoid having replicate on write tasks stacking up at CL.ONE (CASSANDRA-2889)
 + * (cql3) Backwards compatibility for composite comparators in non-cql3-aware
 +   clients (CASSANDRA-4093)
 + * (cql3) Fix order by for reversed queries (CASSANDRA-4160)
 + * (cql3) Add ReversedType support (CASSANDRA-4004)
 + * (cql3) Add timeuuid type (CASSANDRA-4194)
 + * (cql3) Minor fixes (CASSANDRA-4185)
 + * (cql3) Fix prepared statement in BATCH (CASSANDRA-4202)
 + * (cql3) Reduce the list of reserved keywords (CASSANDRA-4186)
 + * (cql3) Move max/min compaction thresholds to compaction strategy options
 +   (CASSANDRA-4187)
 + * Fix exception during move when localhost is the only source (CASSANDRA-4200)
 + * (cql3) Allow paging through non-ordered partitioner results (CASSANDRA-3771)
 + * (cql3) Fix drop index (CASSANDRA-4192)
 + * (cql3) Don't return range ghosts anymore (CASSANDRA-3982)
 + * fix re-creating Keyspaces/ColumnFamilies with the same name as dropped
 +   ones (CASSANDRA-4219)
 + * fix SecondaryIndex LeveledManifest save upon snapshot (CASSANDRA-4230)
 + * fix missing arrayOffset in FBUtilities.hash (CASSANDRA-4250)
 + * (cql3) Add name of parameters in CqlResultSet (CASSANDRA-4242)
 + * (cql3) Correctly validate order by queries (CASSANDRA-4246)
 + * rename stress to cassandra-stress for saner packaging (CASSANDRA-4256)
 + * Fix exception on colum metadata with non-string comparator (CASSANDRA-4269)
 + * Check for unknown/invalid compression options (CASSANDRA-4266)
 + * (cql3) Adds simple access to column timestamp and ttl (CASSANDRA-4217)
 + * (cql3) Fix range queries with secondary indexes (CASSANDRA-4257)
 + * Better error messages from improper input in cli (CASSANDRA-3865)
 + * Try to stop all compaction upon Keyspace or ColumnFamily drop (CASSANDRA-4221)
 + * (cql3) Allow keyspace properties to contain hyphens (CASSANDRA-4278)
 + * (cql3) Correctly validate keyspace access in create table (CASSANDRA-4296)
 + * Avoid deadlock in migration stage (CASSANDRA-3882)
 + * Take supercolumn names and deletion info into account in memtable throughput
 +   (CASSANDRA-4264)
 + * Add back backward compatibility for old style replication factor (CASSANDRA-4294)
 + * Preserve compatibility with pre-1.1 index queries (CASSANDRA-4262)
 +Merged from 1.0:
 + * Fix super columns bug where cache is not updated (CASSANDRA-4190)
 + * fix maxTimestamp to include row tombstones (CASSANDRA-4116)
 + * (CLI) properly handle quotes in create/update keyspace commands (CASSANDRA-4129)
 + * Avoids possible deadlock during bootstrap (CASSANDRA-4159)
 + * fix stress tool that hangs forever on timeout or error (CASSANDRA-4128)
 + * stress tool to return appropriate exit code on failure (CASSANDRA-4188)
 + * fix compaction NPE when out of disk space and assertions disabled
 +   (CASSANDRA-3985)
   * synchronize LCS getEstimatedTasks to avoid CME (CASSANDRA-4255)
   * ensure unique streaming session id's (CASSANDRA-4223)
   * kick off background compaction when min/max thresholds change 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ccd436c0/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 4b39280,2400cfe..0f5be39
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@@ -72,14 -67,18 +72,17 @@@ public class CompactionManager implemen
      private static final Logger logger = LoggerFactory.getLogger(CompactionManager.class);
      public static final CompactionManager instance;
  
+     public static final int NO_GC = Integer.MIN_VALUE;
+     public static final int GC_ALL = Integer.MAX_VALUE;
+ 
      /**
       * compactionLock has two purposes:
 -     * - Compaction acquires its readLock so that multiple compactions can happen simultaneously,
 -     *   but the KS/CF migtations acquire its writeLock, so they can be sure no new SSTables
will
 -     *   be created for a dropped CF posthumously.  (Thus, compaction checks CFS.isValid
while the
 -     *   lock is acquired.)
       * - "Special" compactions will acquire writelock instead of readlock to make sure that
all
 -     *   other compaction activity is quiesced and they can grab ALL the sstables to do
something.
 -     *   TODO this is too big a hammer -- we should only care about quiescing all for the
given CFS.
 +     * other compaction activity is quiesced and they can grab ALL the sstables to do something.
 +     * - Some schema migrations cannot run concurrently with compaction.  (Currently, this
is
 +     *   only when changing compaction strategy -- see CFS.maybeReloadCompactionStrategy.)
 +     *
 +     * TODO this is too big a hammer -- we should only care about quiescing all for the
given CFS.
       */
      private final ReentrantReadWriteLock compactionLock = new ReentrantReadWriteLock();
  
@@@ -830,35 -983,19 +833,35 @@@
  
      private static class ValidationCompactionIterable extends CompactionIterable
      {
 -        public ValidationCompactionIterable(ColumnFamilyStore cfs, Collection<SSTableReader>
sstables, Range range) throws IOException
 +        public ValidationCompactionIterable(ColumnFamilyStore cfs, Collection<SSTableReader>
sstables, Range<Token> range) throws IOException
          {
              super(OperationType.VALIDATION,
 -                  getScanners(sstables, range),
 -                  new CompactionController(cfs, sstables, getDefaultGcBefore(cfs), true));
 +                  cfs.getCompactionStrategy().getScanners(sstables, range),
 +                  new ValidationCompactionController(cfs, sstables));
          }
 +    }
  
 -        protected static List<SSTableScanner> getScanners(Iterable<SSTableReader>
sstables, Range range) throws IOException
 +    /*
 +     * Controller for validation compaction that never purges.
 +     * Note that we should not call cfs.getOverlappingSSTables on the provided
 +     * sstables because those sstables are not guaranteed to be active sstables
 +     * (since we can run repair on a snapshot).
 +     */
 +    private static class ValidationCompactionController extends CompactionController
 +    {
 +        public ValidationCompactionController(ColumnFamilyStore cfs, Collection<SSTableReader>
sstables)
          {
 -            ArrayList<SSTableScanner> scanners = new ArrayList<SSTableScanner>();
 -            for (SSTableReader sstable : sstables)
 -                scanners.add(sstable.getDirectScanner(range));
 -            return scanners;
 +            super(cfs,
-                   Integer.MAX_VALUE,
++                  NO_GC,
 +                  true,
 +                  null,
 +                  cfs.getCompactionStrategy().isKeyExistenceExpensive(ImmutableSet.copyOf(sstables)));
 +        }
 +
 +        @Override
 +        public boolean shouldPurge(DecoratedKey key)
 +        {
 +            return false;
          }
      }
  


Mime
View raw message