cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ifesdj...@apache.org
Subject [5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Date Thu, 15 Jun 2017 17:38:28 GMT
Merge branch 'cassandra-3.0' into cassandra-3.11


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2a0890d0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2a0890d0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2a0890d0

Branch: refs/heads/cassandra-3.11
Commit: 2a0890d0fc5eaaf88d0a2d610a5f500fe943fb92
Parents: 3f725c9 1f54aa4
Author: Alex Petrov <oleksandr.petrov@gmail.com>
Authored: Thu Jun 15 19:35:07 2017 +0200
Committer: Alex Petrov <oleksandr.petrov@gmail.com>
Committed: Thu Jun 15 19:35:07 2017 +0200

----------------------------------------------------------------------
 CHANGES.txt                                     |  1 +
 NEWS.txt                                        | 28 ++++++++
 .../org/apache/cassandra/db/ReadResponse.java   |  9 +--
 .../db/commitlog/CommitLogDescriptor.java       |  2 +-
 .../cassandra/db/filter/ColumnFilter.java       | 70 ++++++++++++++++----
 .../apache/cassandra/net/MessagingService.java  |  7 +-
 .../cassandra/service/MigrationManager.java     | 13 +++-
 .../cassandra/db/filter/ColumnFilterTest.java   | 70 ++++++++++++++++++++
 8 files changed, 178 insertions(+), 22 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2a0890d0/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index 1058c9c,528bbcd..0047c55
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,42 -1,5 +1,43 @@@
 -3.0.14
 +3.11.0
 + * Replace string comparison with regex/number checks in MessagingService test (CASSANDRA-13216)
 + * Fix formatting of duration columns in CQLSH (CASSANDRA-13549)
 + * Fix the problem with duplicated rows when using paging with SASI (CASSANDRA-13302)
 + * Allow CONTAINS statements filtering on the partition key and it’s parts (CASSANDRA-13275)
 + * Fall back to even ranges calculation in clusters with vnodes when tokens are distributed
unevenly (CASSANDRA-13229)
 + * Fix duration type validation to prevent overflow (CASSANDRA-13218)
 + * Forbid unsupported creation of SASI indexes over partition key columns (CASSANDRA-13228)
 + * Reject multiple values for a key in CQL grammar. (CASSANDRA-13369)
 + * UDA fails without input rows (CASSANDRA-13399)
 + * Fix compaction-stress by using daemonInitialization (CASSANDRA-13188)
 + * V5 protocol flags decoding broken (CASSANDRA-13443)
 + * Use write lock not read lock for removing sstables from compaction strategies. (CASSANDRA-13422)
 + * Use corePoolSize equal to maxPoolSize in JMXEnabledThreadPoolExecutors (CASSANDRA-13329)
 + * Avoid rebuilding SASI indexes containing no values (CASSANDRA-12962)
 + * Add charset to Analyser input stream (CASSANDRA-13151)
 + * Fix testLimitSSTables flake caused by concurrent flush (CASSANDRA-12820)
 + * cdc column addition strikes again (CASSANDRA-13382)
 + * Fix static column indexes (CASSANDRA-13277)
 + * DataOutputBuffer.asNewBuffer broken (CASSANDRA-13298)
 + * unittest CipherFactoryTest failed on MacOS (CASSANDRA-13370)
 + * Forbid SELECT restrictions and CREATE INDEX over non-frozen UDT columns (CASSANDRA-13247)
 + * Default logging we ship will incorrectly print "?:?" for "%F:%L" pattern (CASSANDRA-13317)
 + * Possible AssertionError in UnfilteredRowIteratorWithLowerBound (CASSANDRA-13366)
 + * Support unaligned memory access for AArch64 (CASSANDRA-13326)
 + * Improve SASI range iterator efficiency on intersection with an empty range (CASSANDRA-12915).
 + * Fix equality comparisons of columns using the duration type (CASSANDRA-13174)
 + * Obfuscate password in stress-graphs (CASSANDRA-12233)
 + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034)
 + * nodetool stopdaemon errors out (CASSANDRA-13030)
 + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954)
 + * Fix primary index calculation for SASI (CASSANDRA-12910)
 + * More fixes to the TokenAllocator (CASSANDRA-12990)
 + * NoReplicationTokenAllocator should work with zero replication factor (CASSANDRA-12983)
 + * Address message coalescing regression (CASSANDRA-12676)
 + * Delete illegal character from StandardTokenizerImpl.jflex (CASSANDRA-13417)
 + * Fix cqlsh automatic protocol downgrade regression (CASSANDRA-13307)
 + * Tracing payload not passed from QueryMessage to tracing session (CASSANDRA-12835)
 +Merged from 3.0:
+  * Ensure consistent view of partition columns between coordinator and replica in ColumnFilter
(CASSANDRA-13004)
   * Failed unregistering mbean during drop keyspace (CASSANDRA-13346)
   * nodetool scrub/cleanup/upgradesstables exit code is wrong (CASSANDRA-13542)
   * Fix the reported number of sstable data files accessed per read (CASSANDRA-13120)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2a0890d0/NEWS.txt
----------------------------------------------------------------------
diff --cc NEWS.txt
index a56ced6,00ec48d..6bc3388
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -18,6 -18,41 +18,34 @@@ using the provided 'sstableupgrade' too
  
  Upgrading
  ---------
++
++Upgrading
++---------
+    - ALTER TABLE (ADD/DROP COLUMN) operations concurrent with a read might
+      result into data corruption (see CASSANDRA-13004 for more details).
+      Fixing this bug required a messaging protocol version bump. By default,
 -     Cassandra 3.0.14 will use 3014 version for messaging.
++     Cassandra 3.11 will use 3014 version for messaging.
+ 
+      Since Schema Migrations rely the on exact messaging protocol version
+      match between nodes, if you need schema changes during the upgrade
+      process, you have to start your nodes with `-Dcassandra.force_3_0_protocol_version=true`
+      first, in order to temporarily force a backwards compatible protocol.
 -     After the whole cluster is upgraded to 3.0.14, do a rolling
++     After the whole cluster is upgraded to 3.11, do a rolling
+      restart of the cluster without setting that flag.
+ 
 -     3.0.14 nodes with and withouot the flag set will be able to do schema
++     3.11 nodes with and withouot the flag set will be able to do schema
+      migrations with other 3.x and 3.0.x releases.
+ 
 -     While running the cluster with the flag set to true on 3.0.14 (in
++     While running the cluster with the flag set to true on 3.11 (in
+      compatibility mode), avoid adding or removing any columns to/from
+      existing tables.
+ 
+      If your cluster can do without schema migrations during the upgrade
+      time, just start the cluster normally without setting aforementioned
+      flag.
+ 
 -   - If performing a rolling upgrade from 3.0.13, there will be a schema mismatch caused
 -     by a bug with the schema digest calculation in 3.0.13. This will cause unnecessary
 -     but otherwise harmless schema updates, see CASSANDRA-13559 for more details.
 -
 -   - Nothing else specific to this release, but please see previous versions upgrading section,
 -     especially if you are upgrading from 2.2.
 -
 -3.0.13
 -======
 -
 -Upgrading
 ----------
++     If you are upgrading from 3.0.14+ (of 3.0.x branch), you do not have
++     to set an flag while upgrading to ensure schema migrations.
     - The NativeAccessMBean isAvailable method will only return true if the
       native library has been successfully linked. Previously it was returning
       true if JNA could be found but was not taking into account link failures.

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2a0890d0/src/java/org/apache/cassandra/db/ReadResponse.java
----------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2a0890d0/src/java/org/apache/cassandra/db/commitlog/CommitLogDescriptor.java
----------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2a0890d0/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/db/filter/ColumnFilter.java
index 93a848e,c28c0ae..37da86a
--- a/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
+++ b/src/java/org/apache/cassandra/db/filter/ColumnFilter.java
@@@ -62,17 -52,20 +63,16 @@@ public class ColumnFilte
  {
      public static final Serializer serializer = new Serializer();
  
 -    // Distinguish between the 2 cases described above: if 'isFetchAll' is true, then all
columns will be retrieved
 -    // by the query, but the values for column/cells not selected by 'queried' and 'subSelections'
will be skipped.
 -    // Otherwise, only the column/cells returned by 'queried' and 'subSelections' will be
returned at all.
 +    // True if _fetched_ is all the columns, in which case metadata must not be null. If
false,
 +    // then _fetched_ == _queried_ and we only store _queried_.
      private final boolean isFetchAll;
  
-     private final CFMetaData metadata; // can be null if !isFetchAll
- 
 -    private final PartitionColumns queried; // can be null if isFetchAll and we don't want
to skip any value
+     private final PartitionColumns fetched;
 +    private final PartitionColumns queried; // can be null if isFetchAll and _fetched_ ==
_queried_
      private final SortedSetMultimap<ColumnIdentifier, ColumnSubselection> subSelections;
// can be null
  
 -    /**
 -     * Used on replica for deserialisation
 -     */
      private ColumnFilter(boolean isFetchAll,
-                          CFMetaData metadata,
+                          PartitionColumns fetched,
                           PartitionColumns queried,
                           SortedSetMultimap<ColumnIdentifier, ColumnSubselection> subSelections)
      {
@@@ -120,21 -116,10 +120,20 @@@
       */
      public PartitionColumns fetchedColumns()
      {
-         return isFetchAll ? metadata.partitionColumns() : queried;
+         return fetched;
      }
  
 -    public boolean includesAllColumns()
 +    /**
 +     * The columns actually queried by the user.
 +     * <p>
 +     * Note that this is in general not all the columns that are fetched internally (see
{@link #fetchedColumns}).
 +     */
 +    public PartitionColumns queriedColumns()
 +    {
-         assert queried != null || isFetchAll;
-         return queried == null ? metadata.partitionColumns() : queried;
++        return queried == null ? fetched : queried;
 +    }
 +
 +    public boolean fetchesAllColumns()
      {
          return isFetchAll;
      }
@@@ -352,11 -319,28 +351,28 @@@
                      s.put(subSelection.column().name, subSelection);
              }
  
-             return new ColumnFilter(isFetchAll, metadata, queried, s);
 -            return new ColumnFilter(isFetchAll, isFetchAll ? metadata.partitionColumns()
: selectedColumns, selectedColumns, s);
++            return new ColumnFilter(isFetchAll, isFetchAll ? metadata.partitionColumns()
: null, queried, s);
          }
      }
  
      @Override
+     public boolean equals(Object other)
+     {
+         if (other == this)
+             return true;
+ 
+         if (!(other instanceof ColumnFilter))
+             return false;
+ 
+         ColumnFilter otherCf = (ColumnFilter) other;
+ 
+         return otherCf.isFetchAll == this.isFetchAll &&
+                Objects.equals(otherCf.fetched, this.fetched) &&
+                Objects.equals(otherCf.queried, this.queried) &&
+                Objects.equals(otherCf.subSelections, this.subSelections);
 -
+     }
++
+     @Override
      public String toString()
      {
          if (isFetchAll)
@@@ -434,11 -421,27 +456,27 @@@
          {
              int header = in.readUnsignedByte();
              boolean isFetchAll = (header & IS_FETCH_ALL_MASK) != 0;
 -            boolean hasSelection = (header & HAS_SELECTION_MASK) != 0;
 +            boolean hasQueried = (header & HAS_QUERIED_MASK) != 0;
              boolean hasSubSelections = (header & HAS_SUB_SELECTIONS_MASK) != 0;
  
+             PartitionColumns fetched = null;
 -            PartitionColumns selection = null;
 +            PartitionColumns queried = null;
+ 
+             if (isFetchAll)
+             {
+                 if (version >= MessagingService.VERSION_3014)
+                 {
+                     Columns statics = Columns.serializer.deserialize(in, metadata);
+                     Columns regulars = Columns.serializer.deserialize(in, metadata);
+                     fetched = new PartitionColumns(statics, regulars);
+                 }
+                 else
+                 {
+                     fetched = metadata.partitionColumns();
+                 }
+             }
+ 
 -            if (hasSelection)
 +            if (hasQueried)
              {
                  Columns statics = Columns.serializer.deserialize(in, metadata);
                  Columns regulars = Columns.serializer.deserialize(in, metadata);
@@@ -457,7 -460,7 +495,7 @@@
                  }
              }
  
-             return new ColumnFilter(isFetchAll, isFetchAll ? metadata : null, queried, subSelections);
 -            return new ColumnFilter(isFetchAll, fetched, selection, subSelections);
++            return new ColumnFilter(isFetchAll, fetched, queried, subSelections);
          }
  
          public long serializedSize(ColumnFilter selection, int version)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2a0890d0/src/java/org/apache/cassandra/net/MessagingService.java
----------------------------------------------------------------------
diff --cc src/java/org/apache/cassandra/net/MessagingService.java
index 0e81c06,e0f77b7..032dc03
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@@ -86,6 -80,10 +86,10 @@@ import org.apache.cassandra.utils.concu
  
  public final class MessagingService implements MessagingServiceMBean
  {
 -    // Required to allow schema migrations while upgrading within the minor 3.0.x versions
to 3.0.14.
++    // Required to allow schema migrations while upgrading within the minor 3.0.x/3.x versions
to 3.11+.
+     // See CASSANDRA-13004 for details.
+     public final static boolean FORCE_3_0_PROTOCOL_VERSION = Boolean.getBoolean("cassandra.force_3_0_protocol_version");
+ 
      public static final String MBEAN_NAME = "org.apache.cassandra.net:type=MessagingService";
  
      // 8 bits version, so don't waste versions

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2a0890d0/src/java/org/apache/cassandra/service/MigrationManager.java
----------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/cassandra/blob/2a0890d0/test/unit/org/apache/cassandra/db/filter/ColumnFilterTest.java
----------------------------------------------------------------------
diff --cc test/unit/org/apache/cassandra/db/filter/ColumnFilterTest.java
index 0000000,aa56091..db06d20
mode 000000,100644..100644
--- a/test/unit/org/apache/cassandra/db/filter/ColumnFilterTest.java
+++ b/test/unit/org/apache/cassandra/db/filter/ColumnFilterTest.java
@@@ -1,0 -1,70 +1,70 @@@
+ /*
+  * Licensed to the Apache Software Foundation (ASF) under one
+  * or more contributor license agreements.  See the NOTICE file
+  * distributed with this work for additional information
+  * regarding copyright ownership.  The ASF licenses this file
+  * to you under the Apache License, Version 2.0 (the
+  * "License"); you may not use this file except in compliance
+  * with the License.  You may obtain a copy of the License at
+  *
+  *     http://www.apache.org/licenses/LICENSE-2.0
+  *
+  * Unless required by applicable law or agreed to in writing, software
+  * distributed under the License is distributed on an "AS IS" BASIS,
+  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  * See the License for the specific language governing permissions and
+  * limitations under the License.
+  */
+ 
+ package org.apache.cassandra.db.filter;
+ 
+ import org.junit.Test;
+ 
+ import junit.framework.Assert;
+ import org.apache.cassandra.config.CFMetaData;
+ import org.apache.cassandra.config.ColumnDefinition;
+ import org.apache.cassandra.db.marshal.Int32Type;
+ import org.apache.cassandra.dht.Murmur3Partitioner;
+ import org.apache.cassandra.io.util.DataInputBuffer;
+ import org.apache.cassandra.io.util.DataInputPlus;
+ import org.apache.cassandra.io.util.DataOutputBuffer;
+ import org.apache.cassandra.net.MessagingService;
+ import org.apache.cassandra.utils.ByteBufferUtil;
+ 
+ public class ColumnFilterTest
+ {
+     final static ColumnFilter.Serializer serializer = new ColumnFilter.Serializer();
+ 
+     @Test
+     public void columnFilterSerialisationRoundTrip() throws Exception
+     {
+         CFMetaData metadata = CFMetaData.Builder.create("ks", "table")
+                                                 .withPartitioner(Murmur3Partitioner.instance)
+                                                 .addPartitionKey("pk", Int32Type.instance)
+                                                 .addClusteringColumn("ck", Int32Type.instance)
+                                                 .addRegularColumn("v1", Int32Type.instance)
+                                                 .addRegularColumn("v2", Int32Type.instance)
+                                                 .addRegularColumn("v3", Int32Type.instance)
+                                                 .build();
+ 
+         ColumnDefinition v1 = metadata.getColumnDefinition(ByteBufferUtil.bytes("v1"));
+ 
+         testRoundTrip(ColumnFilter.all(metadata), metadata, MessagingService.VERSION_30);
+         testRoundTrip(ColumnFilter.all(metadata), metadata, MessagingService.VERSION_3014);
+ 
+         testRoundTrip(ColumnFilter.selection(metadata.partitionColumns().without(v1)), metadata,
MessagingService.VERSION_30);
+         testRoundTrip(ColumnFilter.selection(metadata.partitionColumns().without(v1)), metadata,
MessagingService.VERSION_3014);
+ 
+         testRoundTrip(ColumnFilter.selection(metadata, metadata.partitionColumns().without(v1)),
metadata, MessagingService.VERSION_30);
+         testRoundTrip(ColumnFilter.selection(metadata, metadata.partitionColumns().without(v1)),
metadata, MessagingService.VERSION_3014);
+     }
+ 
+     static void testRoundTrip(ColumnFilter columnFilter, CFMetaData metadata, int version)
throws Exception
+     {
+         DataOutputBuffer output = new DataOutputBuffer();
+         serializer.serialize(columnFilter, output, version);
+         Assert.assertEquals(serializer.serializedSize(columnFilter, version), output.position());
+         DataInputPlus input = new DataInputBuffer(output.buffer(), false);
+         Assert.assertEquals(serializer.deserialize(input, version, metadata), columnFilter);
+     }
 -}
++}


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message