cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ble...@apache.org
Subject [2/2] cassandra git commit: Merge branch cassandra-2.2 into cassandra-3.0
Date Tue, 27 Sep 2016 12:09:22 GMT
Merge branch cassandra-2.2 into cassandra-3.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b7fc5dc1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b7fc5dc1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b7fc5dc1

Branch: refs/heads/cassandra-3.0
Commit: b7fc5dc1c74b20fc1250c738631487db98b72e50
Parents: b7da003 6dc595d
Author: Benjamin Lerer <b.lerer@gmail.com>
Authored: Tue Sep 27 14:06:08 2016 +0200
Committer: Benjamin Lerer <b.lerer@gmail.com>
Committed: Tue Sep 27 14:08:50 2016 +0200

----------------------------------------------------------------------
 CHANGES.txt                                     |  1 +
 .../apache/cassandra/db/marshal/UserType.java   |  6 +--
 .../cassandra/serializers/ListSerializer.java   | 10 ++++-
 .../cassandra/serializers/MapSerializer.java    | 10 ++++-
 .../cassandra/serializers/SetSerializer.java    | 11 ++++-
 .../cassandra/serializers/UTF8Serializer.java   |  3 ++
 .../validation/entities/CollectionsTest.java    | 42 ++++++++++++++++++++
 .../cql3/validation/entities/TupleTypeTest.java | 10 +++++
 .../cql3/validation/entities/UserTypesTest.java | 11 +++++
 9 files changed, 98 insertions(+), 6 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7fc5dc1/CHANGES.txt
----------------------------------------------------------------------
diff --cc CHANGES.txt
index 0524e49,998849e..576dfb5
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -9,64 -7,6 +9,65 @@@ Merged from 2.2
   * Fix authentication problem when invoking clqsh copy from a SOURCE command (CASSANDRA-12642)
   * Decrement pending range calculator jobs counter in finally block
    (CASSANDRA-12554)
 +Merged from 2.1:
++ * Make Collections deserialization more robust (CASSANDRA-12618)
 + * Add system property to set the max number of native transport requests in queue (CASSANDRA-11363)
 +
 +
 +3.0.9
 + * Handle composite prefixes with final EOC=0 as in 2.x and refactor LegacyLayout.decodeBound
(CASSANDRA-12423)
 + * Fix paging for 2.x to 3.x upgrades (CASSANDRA-11195)
 + * select_distinct_with_deletions_test failing on non-vnode environments (CASSANDRA-11126)
 + * Stack Overflow returned to queries while upgrading (CASSANDRA-12527)
 + * Fix legacy regex for temporary files from 2.2 (CASSANDRA-12565)
 + * Add option to state current gc_grace_seconds to tools/bin/sstablemetadata (CASSANDRA-12208)
 + * Fix file system race condition that may cause LogAwareFileLister to fail to classify
files (CASSANDRA-11889)
 + * Fix file handle leaks due to simultaneous compaction/repair and
 +   listing snapshots, calculating snapshot sizes, or making schema
 +   changes (CASSANDRA-11594)
 + * Fix nodetool repair exits with 0 for some errors (CASSANDRA-12508)
 + * Do not shut down BatchlogManager twice during drain (CASSANDRA-12504)
 + * Disk failure policy should not be invoked on out of space (CASSANDRA-12385)
 + * Calculate last compacted key on startup (CASSANDRA-6216)
 + * Add schema to snapshot manifest, add USING TIMESTAMP clause to ALTER TABLE statements
(CASSANDRA-7190)
 + * Fix clean interval not sent to commit log for empty memtable flush (CASSANDRA-12436)
 + * Fix potential resource leak in RMIServerSocketFactoryImpl (CASSANDRA-12331)
 + * Backport CASSANDRA-12002 (CASSANDRA-12177)
 + * Make sure compaction stats are updated when compaction is interrupted (CASSANDRA-12100)
 + * Fix potential bad messaging service message for paged range reads
 +   within mixed-version 3.x clusters (CASSANDRA-12249)
 + * Change commitlog and sstables to track dirty and clean intervals (CASSANDRA-11828)
 + * NullPointerException during compaction on table with static columns (CASSANDRA-12336)
 + * Fixed ConcurrentModificationException when reading metrics in GraphiteReporter (CASSANDRA-11823)
 + * Fix upgrade of super columns on thrift (CASSANDRA-12335)
 + * Fixed flacky BlacklistingCompactionsTest, switched to fixed size types and increased
corruption size (CASSANDRA-12359)
 + * Rerun ReplicationAwareTokenAllocatorTest on failure to avoid flakiness (CASSANDRA-12277)
 + * Exception when computing read-repair for range tombstones (CASSANDRA-12263)
 + * Lost counter writes in compact table and static columns (CASSANDRA-12219)
 + * AssertionError with MVs on updating a row that isn't indexed due to a null value (CASSANDRA-12247)
 + * Disable RR and speculative retry with EACH_QUORUM reads (CASSANDRA-11980)
 + * Add option to override compaction space check (CASSANDRA-12180)
 + * Faster startup by only scanning each directory for temporary files once (CASSANDRA-12114)
 + * Respond with v1/v2 protocol header when responding to driver that attempts
 +   to connect with too low of a protocol version (CASSANDRA-11464)
 + * NullPointerExpception when reading/compacting table (CASSANDRA-11988)
 + * Fix problem with undeleteable rows on upgrade to new sstable format (CASSANDRA-12144)
 + * Fix paging logic for deleted partitions with static columns (CASSANDRA-12107)
 + * Wait until the message is being send to decide which serializer must be used (CASSANDRA-11393)
 + * Fix migration of static thrift column names with non-text comparators (CASSANDRA-12147)
 + * Fix upgrading sparse tables that are incorrectly marked as dense (CASSANDRA-11315)
 + * Fix reverse queries ignoring range tombstones (CASSANDRA-11733)
 + * Avoid potential race when rebuilding CFMetaData (CASSANDRA-12098)
 + * Avoid missing sstables when getting the canonical sstables (CASSANDRA-11996)
 + * Always select the live sstables when getting sstables in bounds (CASSANDRA-11944)
 + * Fix column ordering of results with static columns for Thrift requests in
 +   a mixed 2.x/3.x cluster, also fix potential non-resolved duplication of
 +   those static columns in query results (CASSANDRA-12123)
 + * Avoid digest mismatch with empty but static rows (CASSANDRA-12090)
 + * Fix EOF exception when altering column type (CASSANDRA-11820)
 + * Fix JsonTransformer output of partition with deletion info (CASSANDRA-12418)
 + * Fix NPE in SSTableLoader when specifying partial directory path (CASSANDRA-12609)
 +Merged from 2.2:
   * Add local address entry in PropertyFileSnitch (CASSANDRA-11332)
   * cqlshlib tests: increase default execute timeout (CASSANDRA-12481)
   * Forward writes to replacement node when replace_address != broadcast_address (CASSANDRA-8523)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7fc5dc1/src/java/org/apache/cassandra/db/marshal/UserType.java
----------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7fc5dc1/src/java/org/apache/cassandra/serializers/ListSerializer.java
----------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7fc5dc1/src/java/org/apache/cassandra/serializers/MapSerializer.java
----------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7fc5dc1/src/java/org/apache/cassandra/serializers/SetSerializer.java
----------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7fc5dc1/test/unit/org/apache/cassandra/cql3/validation/entities/CollectionsTest.java
----------------------------------------------------------------------
diff --cc test/unit/org/apache/cassandra/cql3/validation/entities/CollectionsTest.java
index a0a6e73,115b755..3d26f5a
--- a/test/unit/org/apache/cassandra/cql3/validation/entities/CollectionsTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/entities/CollectionsTest.java
@@@ -690,178 -608,44 +690,220 @@@ public class CollectionsTest extends CQ
      }
  
      @Test
 +    public void testListWithElementsBiggerThan64K() throws Throwable
 +    {
 +        createTable("CREATE TABLE %s (k int PRIMARY KEY, l list<text>)");
 +
 +        byte[] bytes = new byte[FBUtilities.MAX_UNSIGNED_SHORT + 10];
 +        Arrays.fill(bytes, (byte) 1);
 +        String largeText = new String(bytes);
 +
 +        bytes = new byte[FBUtilities.MAX_UNSIGNED_SHORT + 10];
 +        Arrays.fill(bytes, (byte) 2);
 +        String largeText2 = new String(bytes);
 +
 +        execute("INSERT INTO %s(k, l) VALUES (0, ?)", list(largeText, "v2"));
 +        flush();
 +
 +        assertRows(execute("SELECT l FROM %s WHERE k = 0"), row(list(largeText, "v2")));
 +
 +        execute("DELETE l[?] FROM %s WHERE k = 0", 0);
 +
 +        assertRows(execute("SELECT l FROM %s WHERE k = 0"), row(list("v2")));
 +
 +        execute("UPDATE %s SET l[?] = ? WHERE k = 0", 0, largeText);
 +
 +        assertRows(execute("SELECT l FROM %s WHERE k = 0"), row(list(largeText)));
 +
 +        // Full overwrite
 +        execute("UPDATE %s SET l = ? WHERE k = 0", list("v1", largeText));
 +        flush();
 +
 +        assertRows(execute("SELECT l FROM %s WHERE k = 0"), row(list("v1", largeText)));
 +
 +        execute("UPDATE %s SET l = l + ? WHERE k = 0", list("v2", largeText2));
 +
 +        assertRows(execute("SELECT l FROM %s WHERE k = 0"), row(list("v1", largeText, "v2",
largeText2)));
 +
 +        execute("UPDATE %s SET l = l - ? WHERE k = 0", list(largeText, "v2"));
 +
 +        assertRows(execute("SELECT l FROM %s WHERE k = 0"), row(list("v1", largeText2)));
 +
 +        execute("DELETE l FROM %s WHERE k = 0");
 +
 +        assertRows(execute("SELECT l FROM %s WHERE k = 0"), row((Object) null));
 +
 +        execute("INSERT INTO %s(k, l) VALUES (0, ['" + largeText + "', 'v2'])");
 +        flush();
 +
 +        assertRows(execute("SELECT l FROM %s WHERE k = 0"), row(list(largeText, "v2")));
 +    }
 +
 +    @Test
 +    public void testMapsWithElementsBiggerThan64K() throws Throwable
 +    {
 +        byte[] bytes = new byte[FBUtilities.MAX_UNSIGNED_SHORT + 10];
 +        Arrays.fill(bytes, (byte) 1);
 +        String largeText = new String(bytes);
 +        bytes = new byte[FBUtilities.MAX_UNSIGNED_SHORT + 10];
 +        Arrays.fill(bytes, (byte) 2);
 +        String largeText2 = new String(bytes);
 +
 +        createTable("CREATE TABLE %s (k int PRIMARY KEY, m map<text, text>)");
 +
 +        execute("INSERT INTO %s(k, m) VALUES (0, ?)", map("k1", largeText, largeText, "v2"));
 +        flush();
 +
 +        assertRows(execute("SELECT m FROM %s WHERE k = 0"),
 +                   row(map("k1", largeText, largeText, "v2")));
 +
 +        execute("UPDATE %s SET m[?] = ? WHERE k = 0", "k3", largeText);
 +
 +        assertRows(execute("SELECT m FROM %s WHERE k = 0"),
 +                   row(map("k1", largeText, largeText, "v2", "k3", largeText)));
 +
 +        execute("UPDATE %s SET m[?] = ? WHERE k = 0", largeText2, "v4");
 +
 +        assertRows(execute("SELECT m FROM %s WHERE k = 0"),
 +                   row(map("k1", largeText, largeText, "v2", "k3", largeText, largeText2,
"v4")));
 +
 +        execute("DELETE m[?] FROM %s WHERE k = 0", "k1");
 +
 +        assertRows(execute("SELECT m FROM %s WHERE k = 0"),
 +                   row(map(largeText, "v2", "k3", largeText, largeText2, "v4")));
 +
 +        execute("DELETE m[?] FROM %s WHERE k = 0", largeText2);
 +        flush();
 +
 +        assertRows(execute("SELECT m FROM %s WHERE k = 0"),
 +                   row(map(largeText, "v2", "k3", largeText)));
 +
 +        // Full overwrite
 +        execute("UPDATE %s SET m = ? WHERE k = 0", map("k5", largeText, largeText, "v6"));
 +        flush();
 +
 +        assertRows(execute("SELECT m FROM %s WHERE k = 0"),
 +                   row(map("k5", largeText, largeText, "v6")));
 +
 +        execute("UPDATE %s SET m = m + ? WHERE k = 0", map("k7", largeText));
 +
 +        assertRows(execute("SELECT m FROM %s WHERE k = 0"),
 +                   row(map("k5", largeText, largeText, "v6", "k7", largeText)));
 +
 +        execute("UPDATE %s SET m = m + ? WHERE k = 0", map(largeText2, "v8"));
 +        flush();
 +
 +        assertRows(execute("SELECT m FROM %s WHERE k = 0"),
 +                   row(map("k5", largeText, largeText, "v6", "k7", largeText, largeText2,
"v8")));
 +
 +        execute("DELETE m FROM %s WHERE k = 0");
 +
 +        assertRows(execute("SELECT m FROM %s WHERE k = 0"), row((Object) null));
 +
 +        execute("INSERT INTO %s(k, m) VALUES (0, {'" + largeText + "' : 'v1', 'k2' : '"
+ largeText + "'})");
 +        flush();
 +
 +        assertRows(execute("SELECT m FROM %s WHERE k = 0"),
 +                   row(map(largeText, "v1", "k2", largeText)));
 +    }
 +
 +    @Test
 +    public void testSetsWithElementsBiggerThan64K() throws Throwable
 +    {
 +        createTable("CREATE TABLE %s (k int PRIMARY KEY, s set<text>)");
 +
 +        byte[] bytes = new byte[FBUtilities.MAX_UNSIGNED_SHORT + 10];
 +        Arrays.fill(bytes, (byte) 1);
 +        String largeText = new String(bytes);
 +
 +        bytes = new byte[FBUtilities.MAX_UNSIGNED_SHORT + 10];
 +        Arrays.fill(bytes, (byte) 2);
 +        String largeText2 = new String(bytes);
 +
 +        execute("INSERT INTO %s(k, s) VALUES (0, ?)", set(largeText, "v2"));
 +        flush();
 +
 +        assertRows(execute("SELECT s FROM %s WHERE k = 0"), row(set(largeText, "v2")));
 +
 +        execute("DELETE s[?] FROM %s WHERE k = 0", largeText);
 +
 +        assertRows(execute("SELECT s FROM %s WHERE k = 0"), row(set("v2")));
 +
 +        // Full overwrite
 +        execute("UPDATE %s SET s = ? WHERE k = 0", set("v1", largeText));
 +        flush();
 +
 +        assertRows(execute("SELECT s FROM %s WHERE k = 0"), row(set("v1", largeText)));
 +
 +        execute("UPDATE %s SET s = s + ? WHERE k = 0", set("v2", largeText2));
 +
 +        assertRows(execute("SELECT s FROM %s WHERE k = 0"), row(set("v1", largeText, "v2",
largeText2)));
 +
 +        execute("UPDATE %s SET s = s - ? WHERE k = 0", set(largeText, "v2"));
 +
 +        assertRows(execute("SELECT s FROM %s WHERE k = 0"), row(set("v1", largeText2)));
 +
 +        execute("DELETE s FROM %s WHERE k = 0");
 +
 +        assertRows(execute("SELECT s FROM %s WHERE k = 0"), row((Object) null));
 +
 +        execute("INSERT INTO %s(k, s) VALUES (0, {'" + largeText + "', 'v2'})");
 +        flush();
 +
 +        assertRows(execute("SELECT s FROM %s WHERE k = 0"), row(set(largeText, "v2")));
 +    }
 +
 +    @Test
 +    public void testRemovalThroughUpdate() throws Throwable
 +    {
 +        createTable("CREATE TABLE %s (k int PRIMARY KEY, l list<int>)");
 +
 +         execute("INSERT INTO %s(k, l) VALUES(?, ?)", 0, list(1, 2, 3));
 +         assertRows(execute("SELECT * FROM %s"), row(0, list(1, 2, 3)));
 +
 +         execute("UPDATE %s SET l[0] = null WHERE k=0");
 +         assertRows(execute("SELECT * FROM %s"), row(0, list(2, 3)));
 +    }
++
++    @Test
+     public void testInvalidInputForList() throws Throwable
+     {
+         createTable("CREATE TABLE %s(pk int PRIMARY KEY, l list<text>)");
+         assertInvalidMessage("Not enough bytes to read a list",
+                              "INSERT INTO %s (pk, l) VALUES (?, ?)", 1, "test");
+         assertInvalidMessage("Not enough bytes to read a list",
+                              "INSERT INTO %s (pk, l) VALUES (?, ?)", 1, Long.MAX_VALUE);
+         assertInvalidMessage("Not enough bytes to read a list",
+                              "INSERT INTO %s (pk, l) VALUES (?, ?)", 1, "");
+         assertInvalidMessage("The data cannot be deserialized as a list",
+                              "INSERT INTO %s (pk, l) VALUES (?, ?)", 1, -1);
+     }
+ 
+     @Test
+     public void testInvalidInputForSet() throws Throwable
+     {
+         createTable("CREATE TABLE %s(pk int PRIMARY KEY, s set<text>)");
+         assertInvalidMessage("Not enough bytes to read a set",
+                              "INSERT INTO %s (pk, s) VALUES (?, ?)", 1, "test");
+         assertInvalidMessage("String didn't validate.",
+                              "INSERT INTO %s (pk, s) VALUES (?, ?)", 1, Long.MAX_VALUE);
+         assertInvalidMessage("Not enough bytes to read a set",
+                              "INSERT INTO %s (pk, s) VALUES (?, ?)", 1, "");
+         assertInvalidMessage("The data cannot be deserialized as a set",
+                              "INSERT INTO %s (pk, s) VALUES (?, ?)", 1, -1);
+     }
+ 
+     @Test
+     public void testInvalidInputForMap() throws Throwable
+     {
+         createTable("CREATE TABLE %s(pk int PRIMARY KEY, m map<text, text>)");
+         assertInvalidMessage("Not enough bytes to read a map",
+                              "INSERT INTO %s (pk, m) VALUES (?, ?)", 1, "test");
+         assertInvalidMessage("String didn't validate.",
+                              "INSERT INTO %s (pk, m) VALUES (?, ?)", 1, Long.MAX_VALUE);
+         assertInvalidMessage("Not enough bytes to read a map",
+                              "INSERT INTO %s (pk, m) VALUES (?, ?)", 1, "");
+         assertInvalidMessage("The data cannot be deserialized as a map",
+                              "INSERT INTO %s (pk, m) VALUES (?, ?)", 1, -1);
+     }
  }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7fc5dc1/test/unit/org/apache/cassandra/cql3/validation/entities/TupleTypeTest.java
----------------------------------------------------------------------

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7fc5dc1/test/unit/org/apache/cassandra/cql3/validation/entities/UserTypesTest.java
----------------------------------------------------------------------


Mime
View raw message