cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From alek...@apache.org
Subject [06/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Date Mon, 11 Apr 2016 18:19:15 GMT
Merge branch 'cassandra-2.1' into cassandra-2.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ab2b8a60
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ab2b8a60
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ab2b8a60

Branch: refs/heads/cassandra-2.2
Commit: ab2b8a60c4b6d27081d632fefa0e19ee13816e2c
Parents: 3557d2e c1b1d3b
Author: Aleksey Yeschenko <aleksey@apache.org>
Authored: Mon Apr 11 19:14:41 2016 +0100
Committer: Aleksey Yeschenko <aleksey@apache.org>
Committed: Mon Apr 11 19:15:47 2016 +0100

----------------------------------------------------------------------
 CHANGES.txt                                     |  1 +
 conf/cassandra.yaml                             |  3 ++
 .../org/apache/cassandra/config/Config.java     |  1 +
 .../cassandra/config/DatabaseDescriptor.java    |  5 +++
 .../cql3/statements/BatchStatement.java         | 42 +++++---------------
 5 files changed, 21 insertions(+), 31 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab2b8a60/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index e935e57,6385509..419ed21
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,51 -1,9 +1,52 @@@
 -2.1.14
 +2.2.6
 + * Make deprecated repair methods backward-compatible with previous notification service
(CASSANDRA-11430)
 + * IncomingStreamingConnection version check message wrong (CASSANDRA-11462)
 + * DatabaseDescriptor should log stacktrace in case of Eception during seed provider creation
(CASSANDRA-11312)
 + * Use canonical path for directory in SSTable descriptor (CASSANDRA-10587)
 + * Add cassandra-stress keystore option (CASSANDRA-9325)
 + * Fix out-of-space error treatment in memtable flushing (CASSANDRA-11448).
 + * Dont mark sstables as repairing with sub range repairs (CASSANDRA-11451)
 + * Fix use of NullUpdater for 2i during compaction (CASSANDRA-11450)
 + * Notify when sstables change after cancelling compaction (CASSANDRA-11373)
 + * cqlsh: COPY FROM should check that explicit column names are valid (CASSANDRA-11333)
 + * Add -Dcassandra.start_gossip startup option (CASSANDRA-10809)
 + * Fix UTF8Validator.validate() for modified UTF-8 (CASSANDRA-10748)
 + * Clarify that now() function is calculated on the coordinator node in CQL documentation
(CASSANDRA-10900)
 + * Fix bloom filter sizing with LCS (CASSANDRA-11344)
 + * (cqlsh) Fix error when result is 0 rows with EXPAND ON (CASSANDRA-11092)
 + * Fix intra-node serialization issue for multicolumn-restrictions (CASSANDRA-11196)
 + * Non-obsoleting compaction operations over compressed files can impose rate limit on normal
reads (CASSANDRA-11301)
 + * Add missing newline at end of bin/cqlsh (CASSANDRA-11325)
 + * Fix AE in nodetool cfstats (backport CASSANDRA-10859) (CASSANDRA-11297)
 + * Unresolved hostname leads to replace being ignored (CASSANDRA-11210)
 + * Fix filtering on non-primary key columns for thrift static column families
 +   (CASSANDRA-6377)
 + * Only log yaml config once, at startup (CASSANDRA-11217)
 + * Preserve order for preferred SSL cipher suites (CASSANDRA-11164)
 + * Reference leak with parallel repairs on the same table (CASSANDRA-11215)
 + * Range.compareTo() violates the contract of Comparable (CASSANDRA-11216)
 + * Avoid NPE when serializing ErrorMessage with null message (CASSANDRA-11167)
 + * Replacing an aggregate with a new version doesn't reset INITCOND (CASSANDRA-10840)
 + * (cqlsh) cqlsh cannot be called through symlink (CASSANDRA-11037)
 + * fix ohc and java-driver pom dependencies in build.xml (CASSANDRA-10793)
 + * Protect from keyspace dropped during repair (CASSANDRA-11065)
 + * Handle adding fields to a UDT in SELECT JSON and toJson() (CASSANDRA-11146)
 + * Better error message for cleanup (CASSANDRA-10991)
 + * cqlsh pg-style-strings broken if line ends with ';' (CASSANDRA-11123)
 + * Use cloned TokenMetadata in size estimates to avoid race against membership check
 +   (CASSANDRA-10736)
 + * Always persist upsampled index summaries (CASSANDRA-10512)
 + * (cqlsh) Fix inconsistent auto-complete (CASSANDRA-10733)
 + * Make SELECT JSON and toJson() threadsafe (CASSANDRA-11048)
 + * Fix SELECT on tuple relations for mixed ASC/DESC clustering order (CASSANDRA-7281)
 + * (cqlsh) Support utf-8/cp65001 encoding on Windows (CASSANDRA-11030)
 + * Fix paging on DISTINCT queries repeats result when first row in partition changes
 +   (CASSANDRA-10010)
 +Merged from 2.1:
+  * Checking if an unlogged batch is local is inefficient (CASSANDRA-11529)
   * Fix paging for COMPACT tables without clustering columns (CASSANDRA-11467)
 - * Fix out-of-space error treatment in memtable flushing (CASSANDRA-11448)
 - * Backport CASSANDRA-10859 (CASSANDRA-11415)
 - * COPY FROM fails when importing blob (CASSANDRA-11375)
 + * Add a -j parameter to scrub/cleanup/upgradesstables to state how
 +   many threads to use (CASSANDRA-11179)
   * Backport CASSANDRA-10679 (CASSANDRA-9598)
   * Don't do defragmentation if reading from repaired sstables (CASSANDRA-10342)
   * Fix streaming_socket_timeout_in_ms not enforced (CASSANDRA-11286)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab2b8a60/conf/cassandra.yaml
----------------------------------------------------------------------
diff --cc conf/cassandra.yaml
index 7e86acd,90c5be4..7c46f09
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@@ -593,9 -555,10 +593,12 @@@ column_index_size_in_kb: 6
  # Caution should be taken on increasing the size of this threshold as it can lead to node
instability.
  batch_size_warn_threshold_in_kb: 5
  
 +# Fail any batch exceeding this value. 50kb (10x warn threshold) by default.
 +batch_size_fail_threshold_in_kb: 50
  
+ # Log WARN on any batches not of type LOGGED than span across more partitions than this
limit
+ unlogged_batch_across_partitions_warn_threshold: 10
+ 
  # Number of simultaneous compactions to allow, NOT including
  # validation "compactions" for anti-entropy repair.  Simultaneous
  # compactions can help preserve read performance in a mixed read/write

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab2b8a60/src/java/org/apache/cassandra/config/Config.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/config/Config.java
index be5f175,9ff7096..6116a66
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@@ -164,8 -143,8 +164,9 @@@ public class Confi
  
      /* if the size of columns or super-columns are more than this, indexing will kick in
*/
      public Integer column_index_size_in_kb = 64;
 -    public Integer batch_size_warn_threshold_in_kb = 5;
 +    public volatile int batch_size_warn_threshold_in_kb = 5;
 +    public volatile int batch_size_fail_threshold_in_kb = 50;
+     public Integer unlogged_batch_across_partitions_warn_threshold = 10;
      public Integer concurrent_compactors;
      public volatile Integer compaction_throughput_mb_per_sec = 16;
      public volatile Integer compaction_large_partition_warning_threshold_mb = 100;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab2b8a60/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 46e832d,166ce7e..e24917c
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@@ -861,26 -860,11 +861,31 @@@ public class DatabaseDescripto
          return conf.batch_size_warn_threshold_in_kb * 1024;
      }
  
 +    public static long getBatchSizeFailThreshold()
 +    {
 +        return conf.batch_size_fail_threshold_in_kb * 1024L;
 +    }
 +
 +    public static int getBatchSizeFailThresholdInKB()
 +    {
 +        return conf.batch_size_fail_threshold_in_kb;
 +    }
 +
+     public static int getUnloggedBatchAcrossPartitionsWarnThreshold()
+     {
+         return conf.unlogged_batch_across_partitions_warn_threshold;
+     }
+ 
 +    public static void setBatchSizeWarnThresholdInKB(int threshold)
 +    {
 +        conf.batch_size_warn_threshold_in_kb = threshold;
 +    }
 +
 +    public static void setBatchSizeFailThresholdInKB(int threshold)
 +    {
 +        conf.batch_size_fail_threshold_in_kb = threshold;
 +    }
 +
      public static Collection<String> getInitialTokens()
      {
          return tokensFromString(System.getProperty("cassandra.initial_token", conf.initial_token));

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ab2b8a60/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
index a289ad1,ada8d91..76e389b
--- a/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/BatchStatement.java
@@@ -33,15 -33,10 +33,12 @@@ import org.apache.cassandra.config.Data
  import org.apache.cassandra.cql3.*;
  import org.apache.cassandra.db.*;
  import org.apache.cassandra.db.composites.Composite;
- import org.apache.cassandra.dht.Range;
- import org.apache.cassandra.dht.Token;
  import org.apache.cassandra.exceptions.*;
  import org.apache.cassandra.service.ClientState;
 +import org.apache.cassandra.service.ClientWarn;
  import org.apache.cassandra.service.QueryState;
  import org.apache.cassandra.service.StorageProxy;
- import org.apache.cassandra.service.StorageService;
 +import org.apache.cassandra.tracing.Tracing;
  import org.apache.cassandra.transport.messages.ResultMessage;
  import org.apache.cassandra.utils.NoSpamLogger;
  import org.apache.cassandra.utils.Pair;
@@@ -287,32 -264,18 +285,25 @@@ public class BatchStatement implements 
              for (IMutation im : mutations)
              {
                  keySet.add(im.key());
+ 
                  for (ColumnFamily cf : im.getColumnFamilies())
                      ksCfPairs.add(String.format("%s.%s", cf.metadata().ksName, cf.metadata().cfName));
- 
-                 if (localMutationsOnly)
-                     localMutationsOnly &= isMutationLocal(localTokensByKs, im);
              }
  
-             // CASSANDRA-9303: If we only have local mutations we do not warn
-             if (localMutationsOnly)
-                 return;
- 
-             NoSpamLogger.log(logger, NoSpamLogger.Level.WARN, 1, TimeUnit.MINUTES, unloggedBatchWarning,
-                              keySet.size(), keySet.size() == 1 ? "" : "s",
-                              ksCfPairs.size() == 1 ? "" : "s", ksCfPairs);
+             // CASSANDRA-11529: log only if we have more than a threshold of keys, this
was also suggested in the
+             // original ticket that introduced this warning, CASSANDRA-9282
+             if (keySet.size() > DatabaseDescriptor.getUnloggedBatchAcrossPartitionsWarnThreshold())
+             {
+                 NoSpamLogger.log(logger, NoSpamLogger.Level.WARN, 1, TimeUnit.MINUTES, unloggedBatchWarning,
+                                  keySet.size(), ksCfPairs.size() == 1 ? "" : "s", ksCfPairs);
 +
-             ClientWarn.instance.warn(MessageFormatter.arrayFormat(unloggedBatchWarning,
++                ClientWarn.instance.warn(MessageFormatter.arrayFormat(unloggedBatchWarning,
 +                                                                  new Object[]{
 +                                                                              keySet.size(),
-                                                                               keySet.size()
== 1 ? "" : "s",
 +                                                                              ksCfPairs.size()
== 1 ? "" : "s",
 +                                                                              ksCfPairs
 +                                                                  }).getMessage());
- 
+             }
          }
      }
  


Mime
View raw message