cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From slebre...@apache.org
Subject cassandra git commit: Option to leave omitted columns in INSERT JSON unset
Date Tue, 19 Jul 2016 13:49:23 GMT
Repository: cassandra
Updated Branches:
  refs/heads/trunk d314b6057 -> 12911352d


Option to leave omitted columns in INSERT JSON unset

patch by Oded Peer; reviewed by Sylvain Lebresne for CASSANDRA-11424


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/12911352
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/12911352
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/12911352

Branch: refs/heads/trunk
Commit: 12911352de32c948282779a70848211008409441
Parents: d314b60
Author: Sylvain Lebresne <sylvain@datastax.com>
Authored: Mon Jul 18 16:05:31 2016 +0200
Committer: Sylvain Lebresne <sylvain@datastax.com>
Committed: Tue Jul 19 15:49:09 2016 +0200

----------------------------------------------------------------------
 CHANGES.txt                                     |  1 +
 NEWS.txt                                        |  5 +--
 doc/source/cql/changes.rst                      |  5 +++
 doc/source/cql/dml.rst                          |  2 +-
 doc/source/cql/json.rst                         |  7 ++-
 src/antlr/Lexer.g                               |  2 +
 src/antlr/Parser.g                              |  4 +-
 .../org/apache/cassandra/cql3/Constants.java    | 26 +++++++++++
 src/java/org/apache/cassandra/cql3/Json.java    | 34 ++++++++++-----
 .../cql3/statements/UpdateStatement.java        |  6 ++-
 .../cql3/validation/entities/JsonTest.java      | 46 ++++++++++++++++++++
 11 files changed, 119 insertions(+), 19 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/CHANGES.txt
----------------------------------------------------------------------
diff --git a/CHANGES.txt b/CHANGES.txt
index 7d30aa4..1a32c63 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.10
+ * Option to leave omitted columns in INSERT JSON unset (CASSANDRA-11424)
  * Support json/yaml output in nodetool tpstats (CASSANDRA-12035)
  * Expose metrics for successful/failed authentication attempts (CASSANDRA-10635)
  * Prepend snapshot name with "truncated" or "dropped" when a snapshot

http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/NEWS.txt
----------------------------------------------------------------------
diff --git a/NEWS.txt b/NEWS.txt
index 99948fe..0492b25 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -23,18 +23,17 @@ New features
      the system keyspace. Upon startup, this table is used to preload all
      previously prepared statements - i.e. in many cases clients do not need to
      re-prepare statements against restarted nodes.
-
    - cqlsh can now connect to older Cassandra versions by downgrading the native
      protocol version. Please note that this is currently not part of our release
      testing and, as a consequence, it is not guaranteed to work in all cases.
      See CASSANDRA-12150 for more details.
-
    - Snapshots that are automatically taken before a table is dropped or truncated
      will have a "dropped" or "truncated" prefix on their snapshot tag name.
-
    - Metrics are exposed for successful and failed authentication attempts.
      These can be located using the object names org.apache.cassandra.metrics:type=Client,name=AuthSuccess
      and org.apache.cassandra.metrics:type=Client,name=AuthFailure respectively.
+   - Add support to "unset" JSON fields in prepared statements by specifying DEFAULT UNSET.
+     See CASSANDRA-11424 for details
 
 Upgrading
 ---------

http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/doc/source/cql/changes.rst
----------------------------------------------------------------------
diff --git a/doc/source/cql/changes.rst b/doc/source/cql/changes.rst
index d9aea85..d0c51cc 100644
--- a/doc/source/cql/changes.rst
+++ b/doc/source/cql/changes.rst
@@ -21,6 +21,11 @@ Changes
 
 The following describes the changes in each version of CQL.
 
+3.4.3
+^^^^^
+
+- Adds a ``DEFAULT UNSET`` option for ``INSERT JSON`` to ignore omitted columns (:jira:`11424`).
+
 3.4.2
 ^^^^^
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/doc/source/cql/dml.rst
----------------------------------------------------------------------
diff --git a/doc/source/cql/dml.rst b/doc/source/cql/dml.rst
index b5f9e9f..f1c126b 100644
--- a/doc/source/cql/dml.rst
+++ b/doc/source/cql/dml.rst
@@ -295,7 +295,7 @@ Inserting data for a row is done using an ``INSERT`` statement:
                    : [ IF NOT EXISTS ]
                    : [ USING `update_parameter` ( AND `update_parameter` )* ]
    names_values: `names` VALUES `tuple_literal`
-   json_clause: JSON `string`
+   json_clause: JSON `string` [ DEFAULT ( NULL | UNSET ) ]
    names: '(' `column_name` ( ',' `column_name` )* ')'
 
 For instance::

http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/doc/source/cql/json.rst
----------------------------------------------------------------------
diff --git a/doc/source/cql/json.rst b/doc/source/cql/json.rst
index f83f16c..539180a 100644
--- a/doc/source/cql/json.rst
+++ b/doc/source/cql/json.rst
@@ -49,8 +49,11 @@ table with two columns named "myKey" and "value", you would do the following::
 
     INSERT INTO mytable JSON '{ "\"myKey\"": 0, "value": 0}'
 
-Any columns which are omitted from the ``JSON`` map will be defaulted to a ``NULL`` value
(which will result in a
-tombstone being created).
+By default (or if ``DEFAULT NULL`` is explicitly used), a column omitted from the ``JSON``
map will be set to ``NULL``,
+meaning that any pre-existing value for that column will be removed (resulting in a tombstone
being created).
+Alternatively, if the ``DEFAULT UNSET`` directive is used after the value, omitted column
values will be left unset,
+meaning that pre-existing values for those column will be preserved.
+
 
 JSON Encoding of Cassandra Data Types
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/src/antlr/Lexer.g
----------------------------------------------------------------------
diff --git a/src/antlr/Lexer.g b/src/antlr/Lexer.g
index 16b2ac4..a65b4f5 100644
--- a/src/antlr/Lexer.g
+++ b/src/antlr/Lexer.g
@@ -195,6 +195,8 @@ K_OR:          O R;
 K_REPLACE:     R E P L A C E;
 
 K_JSON:        J S O N;
+K_DEFAULT:     D E F A U L T;
+K_UNSET:       U N S E T;
 K_LIKE:        L I K E;
 
 // Case-insensitive alpha characters

http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/src/antlr/Parser.g
----------------------------------------------------------------------
diff --git a/src/antlr/Parser.g b/src/antlr/Parser.g
index f61f464..f00f9d0 100644
--- a/src/antlr/Parser.g
+++ b/src/antlr/Parser.g
@@ -359,12 +359,14 @@ jsonInsertStatement [CFName cf] returns [UpdateStatement.ParsedInsertJson
expr]
     @init {
         Attributes.Raw attrs = new Attributes.Raw();
         boolean ifNotExists = false;
+        boolean defaultUnset = false;
     }
     : val=jsonValue
+      ( K_DEFAULT ( K_NULL | ( { defaultUnset = true; } K_UNSET) ) )?
       ( K_IF K_NOT K_EXISTS { ifNotExists = true; } )?
       ( usingClause[attrs] )?
       {
-          $expr = new UpdateStatement.ParsedInsertJson(cf, attrs, val, ifNotExists);
+          $expr = new UpdateStatement.ParsedInsertJson(cf, attrs, val, defaultUnset, ifNotExists);
       }
     ;
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/src/java/org/apache/cassandra/cql3/Constants.java
----------------------------------------------------------------------
diff --git a/src/java/org/apache/cassandra/cql3/Constants.java b/src/java/org/apache/cassandra/cql3/Constants.java
index 913ea97..f108e8b 100644
--- a/src/java/org/apache/cassandra/cql3/Constants.java
+++ b/src/java/org/apache/cassandra/cql3/Constants.java
@@ -40,6 +40,32 @@ public abstract class Constants
         STRING, INTEGER, UUID, FLOAT, BOOLEAN, HEX;
     }
 
+    private static class UnsetLiteral extends Term.Raw
+    {
+        public Term prepare(String keyspace, ColumnSpecification receiver) throws InvalidRequestException
+        {
+            return UNSET_VALUE;
+        }
+
+        public AssignmentTestable.TestResult testAssignment(String keyspace, ColumnSpecification
receiver)
+        {
+            return AssignmentTestable.TestResult.NOT_ASSIGNABLE;
+        }
+
+        public String getText()
+        {
+            return "";
+        }
+
+        public AbstractType<?> getExactTypeIfKnown(String keyspace)
+        {
+            return null;
+        }
+    }
+
+    // We don't have "unset" literal in the syntax, but it's used implicitely for JSON "DEFAULT
UNSET" option
+    public static final UnsetLiteral UNSET_LITERAL = new UnsetLiteral();
+
     public static final Value UNSET_VALUE = new Value(ByteBufferUtil.UNSET_BYTE_BUFFER);
 
     private static class NullLiteral extends Term.Raw

http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/src/java/org/apache/cassandra/cql3/Json.java
----------------------------------------------------------------------
diff --git a/src/java/org/apache/cassandra/cql3/Json.java b/src/java/org/apache/cassandra/cql3/Json.java
index c018f24..2e67a1e 100644
--- a/src/java/org/apache/cassandra/cql3/Json.java
+++ b/src/java/org/apache/cassandra/cql3/Json.java
@@ -111,7 +111,7 @@ public class Json
      */
     public static abstract class Prepared
     {
-        public abstract Term.Raw getRawTermForColumn(ColumnDefinition def);
+        public abstract Term.Raw getRawTermForColumn(ColumnDefinition def, boolean defaultUnset);
     }
 
     /**
@@ -126,10 +126,12 @@ public class Json
             this.columnMap = columnMap;
         }
 
-        public Term.Raw getRawTermForColumn(ColumnDefinition def)
+        public Term.Raw getRawTermForColumn(ColumnDefinition def, boolean defaultUnset)
         {
             Term value = columnMap.get(def.name);
-            return value == null ? Constants.NULL_LITERAL : new ColumnValue(value);
+            return value == null
+                 ? (defaultUnset ? Constants.UNSET_LITERAL : Constants.NULL_LITERAL)
+                 : new ColumnValue(value);
         }
     }
 
@@ -147,9 +149,9 @@ public class Json
             this.columns = columns;
         }
 
-        public RawDelayedColumnValue getRawTermForColumn(ColumnDefinition def)
+        public RawDelayedColumnValue getRawTermForColumn(ColumnDefinition def, boolean defaultUnset)
         {
-            return new RawDelayedColumnValue(this, def);
+            return new RawDelayedColumnValue(this, def, defaultUnset);
         }
     }
 
@@ -198,17 +200,19 @@ public class Json
     {
         private final PreparedMarker marker;
         private final ColumnDefinition column;
+        private final boolean defaultUnset;
 
-        public RawDelayedColumnValue(PreparedMarker prepared, ColumnDefinition column)
+        public RawDelayedColumnValue(PreparedMarker prepared, ColumnDefinition column, boolean
defaultUnset)
         {
             this.marker = prepared;
             this.column = column;
+            this.defaultUnset = defaultUnset;
         }
 
         @Override
         public Term prepare(String keyspace, ColumnSpecification receiver) throws InvalidRequestException
         {
-            return new DelayedColumnValue(marker, column);
+            return new DelayedColumnValue(marker, column, defaultUnset);
         }
 
         @Override
@@ -235,11 +239,13 @@ public class Json
     {
         private final PreparedMarker marker;
         private final ColumnDefinition column;
+        private final boolean defaultUnset;
 
-        public DelayedColumnValue(PreparedMarker prepared, ColumnDefinition column)
+        public DelayedColumnValue(PreparedMarker prepared, ColumnDefinition column, boolean
defaultUnset)
         {
             this.marker = prepared;
             this.column = column;
+            this.defaultUnset = defaultUnset;
         }
 
         @Override
@@ -258,7 +264,9 @@ public class Json
         public Terminal bind(QueryOptions options) throws InvalidRequestException
         {
             Term term = options.getJsonColumnValue(marker.bindIndex, column.name, marker.columns);
-            return term == null ? null : term.bind(options);
+            return term == null
+                 ? (defaultUnset ? Constants.UNSET_VALUE : null)
+                 : term.bind(options);
         }
 
         @Override
@@ -284,10 +292,16 @@ public class Json
             Map<ColumnIdentifier, Term> columnMap = new HashMap<>(expectedReceivers.size());
             for (ColumnSpecification spec : expectedReceivers)
             {
+                // We explicitely test containsKey() because the value itself can be null,
and we want to distinguish an
+                // explicit null value from no value
+                if (!valueMap.containsKey(spec.name.toString()))
+                    continue;
+
                 Object parsedJsonObject = valueMap.remove(spec.name.toString());
                 if (parsedJsonObject == null)
                 {
-                    columnMap.put(spec.name, null);
+                    // This is an explicit user null
+                    columnMap.put(spec.name, Constants.NULL_VALUE);
                 }
                 else
                 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/src/java/org/apache/cassandra/cql3/statements/UpdateStatement.java
----------------------------------------------------------------------
diff --git a/src/java/org/apache/cassandra/cql3/statements/UpdateStatement.java b/src/java/org/apache/cassandra/cql3/statements/UpdateStatement.java
index 3657f94..6bcfd9c 100644
--- a/src/java/org/apache/cassandra/cql3/statements/UpdateStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/UpdateStatement.java
@@ -203,11 +203,13 @@ public class UpdateStatement extends ModificationStatement
     public static class ParsedInsertJson extends ModificationStatement.Parsed
     {
         private final Json.Raw jsonValue;
+        private final boolean defaultUnset;
 
-        public ParsedInsertJson(CFName name, Attributes.Raw attrs, Json.Raw jsonValue, boolean
ifNotExists)
+        public ParsedInsertJson(CFName name, Attributes.Raw attrs, Json.Raw jsonValue, boolean
defaultUnset, boolean ifNotExists)
         {
             super(name, StatementType.INSERT, attrs, null, ifNotExists, false);
             this.jsonValue = jsonValue;
+            this.defaultUnset = defaultUnset;
         }
 
         @Override
@@ -230,7 +232,7 @@ public class UpdateStatement extends ModificationStatement
                 if (def.isClusteringColumn())
                     hasClusteringColumnsSet = true;
 
-                Term.Raw raw = prepared.getRawTermForColumn(def);
+                Term.Raw raw = prepared.getRawTermForColumn(def, defaultUnset);
                 if (def.isPrimaryKeyColumn())
                 {
                     whereClause.add(new SingleColumnRelation(ColumnDefinition.Raw.forColumn(def),
Operator.EQ, raw));

http://git-wip-us.apache.org/repos/asf/cassandra/blob/12911352/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java
----------------------------------------------------------------------
diff --git a/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java b/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java
index 0e255e4..a14e4a5 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java
@@ -794,6 +794,52 @@ public class JsonTest extends CQLTester
     }
 
     @Test
+    public void testInsertJsonSyntaxDefaultUnset() throws Throwable
+    {
+        createTable("CREATE TABLE %s (k int primary key, v1 int, v2 int)");
+        execute("INSERT INTO %s JSON ?", "{\"k\": 0, \"v1\": 0, \"v2\": 0}");
+
+        // leave v1 unset
+        execute("INSERT INTO %s JSON ? DEFAULT UNSET", "{\"k\": 0, \"v2\": 2}");
+        assertRows(execute("SELECT * FROM %s"),
+                row(0, 0, 2)
+        );
+
+        // explicit specification DEFAULT NULL
+        execute("INSERT INTO %s JSON ? DEFAULT NULL", "{\"k\": 0, \"v2\": 2}");
+        assertRows(execute("SELECT * FROM %s"),
+                row(0, null, 2)
+        );
+
+        // implicitly setting v2 to null
+        execute("INSERT INTO %s JSON ? DEFAULT NULL", "{\"k\": 0}");
+        assertRows(execute("SELECT * FROM %s"),
+                row(0, null, null)
+        );
+
+        // mix setting null explicitly with default unset:
+        // set values for all fields
+        execute("INSERT INTO %s JSON ?", "{\"k\": 1, \"v1\": 1, \"v2\": 1}");
+        // explicitly set v1 to null while leaving v2 unset which retains its value
+        execute("INSERT INTO %s JSON ? DEFAULT UNSET", "{\"k\": 1, \"v1\": null}");
+        assertRows(execute("SELECT * FROM %s WHERE k=1"),
+                row(1, null, 1)
+        );
+
+        // test string literal instead of bind marker
+        execute("INSERT INTO %s JSON '{\"k\": 2, \"v1\": 2, \"v2\": 2}'");
+        // explicitly set v1 to null while leaving v2 unset which retains its value
+        execute("INSERT INTO %s JSON '{\"k\": 2, \"v1\": null}' DEFAULT UNSET");
+        assertRows(execute("SELECT * FROM %s WHERE k=2"),
+                row(2, null, 2)
+        );
+        execute("INSERT INTO %s JSON '{\"k\": 2}' DEFAULT NULL");
+        assertRows(execute("SELECT * FROM %s WHERE k=2"),
+                row(2, null, null)
+        );
+    }
+
+    @Test
     public void testCaseSensitivity() throws Throwable
     {
         createTable("CREATE TABLE %s (k int primary key, \"Foo\" int)");


Mime
View raw message