Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id BFC82200BA6 for ; Tue, 18 Oct 2016 14:15:15 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id BE59B160ACC; Tue, 18 Oct 2016 12:15:15 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 95224160ADC for ; Tue, 18 Oct 2016 14:15:13 +0200 (CEST) Received: (qmail 41460 invoked by uid 500); 18 Oct 2016 12:15:12 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 41369 invoked by uid 99); 18 Oct 2016 12:15:12 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 Oct 2016 12:15:12 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 51113E049D; Tue, 18 Oct 2016 12:15:12 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: blerer@apache.org To: commits@cassandra.apache.org Date: Tue, 18 Oct 2016 12:15:12 -0000 Message-Id: <6bd6cf6b31fd4b368482fe8a79e10d15@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [1/2] cassandra git commit: Add duration data type archived-at: Tue, 18 Oct 2016 12:15:15 -0000 Repository: cassandra Updated Branches: refs/heads/trunk bfd57d13b -> b0fdab4e4 Add duration data type patch by Benjamin Lerer; reviewed by Tyler Hobbs for CASSANDRA-11873 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ecf05b88 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ecf05b88 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ecf05b88 Branch: refs/heads/trunk Commit: ecf05b882658d78f0ce6b87b57c982aa776c5104 Parents: c6ec31b Author: Benjamin Lerer Authored: Tue Oct 18 13:47:05 2016 +0200 Committer: Benjamin Lerer Committed: Tue Oct 18 14:01:05 2016 +0200 ---------------------------------------------------------------------- CHANGES.txt | 1 + NEWS.txt | 3 + doc/cql3/CQL.textile | 1 - doc/source/cql/changes.rst | 1 + doc/source/cql/types.rst | 42 ++ pylib/cqlshlib/cql3handling.py | 4 +- pylib/cqlshlib/displaying.py | 1 + pylib/cqlshlib/formatting.py | 82 +++ src/antlr/Lexer.g | 22 + src/antlr/Parser.g | 2 + .../org/apache/cassandra/cql3/CQL3Type.java | 17 + .../org/apache/cassandra/cql3/Constants.java | 15 +- .../org/apache/cassandra/cql3/Duration.java | 590 +++++++++++++++++++ .../cassandra/cql3/SingleColumnRelation.java | 3 + .../cql3/statements/CreateTableStatement.java | 4 + .../cql3/statements/CreateViewStatement.java | 5 +- .../cassandra/db/marshal/AbstractType.java | 5 + .../cassandra/db/marshal/DurationType.java | 95 +++ .../apache/cassandra/db/marshal/ListType.java | 6 + .../apache/cassandra/db/marshal/MapType.java | 7 + .../apache/cassandra/db/marshal/TupleType.java | 6 + .../apache/cassandra/db/marshal/UserType.java | 6 + .../serializers/DurationSerializer.java | 94 +++ .../apache/cassandra/config/CFMetaDataTest.java | 10 +- .../cassandra/cql3/CQL3TypeLiteralTest.java | 7 + .../org/apache/cassandra/cql3/CQLTester.java | 3 + .../org/apache/cassandra/cql3/DurationTest.java | 114 ++++ .../org/apache/cassandra/cql3/ViewTest.java | 21 + .../validation/entities/CollectionsTest.java | 6 +- .../cql3/validation/entities/JsonTest.java | 25 +- .../cql3/validation/operations/BatchTest.java | 8 +- .../cql3/validation/operations/CreateTest.java | 131 ++++ 32 files changed, 1315 insertions(+), 22 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/CHANGES.txt ---------------------------------------------------------------------- diff --git a/CHANGES.txt b/CHANGES.txt index 32a2dfd..c0bc0e4 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.10 + * Add duration data type (CASSANDRA-11873) * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784) * Improve sum aggregate functions (CASSANDRA-12417) * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in CASSANDRA-10876 (CASSANDRA-12761) http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/NEWS.txt ---------------------------------------------------------------------- diff --git a/NEWS.txt b/NEWS.txt index ad0f2be..63c7e6b 100644 --- a/NEWS.txt +++ b/NEWS.txt @@ -18,6 +18,7 @@ using the provided 'sstableupgrade' tool. New features ------------ + - New `DurationType` (cql duration). See CASSANDRA-11873 - Runtime modification of concurrent_compactors is now available via nodetool - Support for the assignment operators +=/-= has been added for update queries. - An Index implementation may now provide a task which runs prior to joining @@ -98,6 +99,8 @@ Upgrading - Application layer keep-alives were added to the streaming protocol to prevent idle incoming connections from timing out and failing the stream session (CASSANDRA-11839). This effectively deprecates the streaming_socket_timeout_in_ms property in favor of streaming_keep_alive_period_in_secs. See cassandra.yaml for more details about this property. + - Duration litterals support the ISO 8601 format. By consequence, identifiers matching that format + (e.g P2Y or P1MT6H) will not be supported anymore (CASSANDRA-11873). 3.8 === http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/doc/cql3/CQL.textile ---------------------------------------------------------------------- diff --git a/doc/cql3/CQL.textile b/doc/cql3/CQL.textile index 07c8c61..7d887db 100644 --- a/doc/cql3/CQL.textile +++ b/doc/cql3/CQL.textile @@ -1,4 +1,3 @@ - h1. Cassandra Query Language (CQL) v3.4.3 http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/doc/source/cql/changes.rst ---------------------------------------------------------------------- diff --git a/doc/source/cql/changes.rst b/doc/source/cql/changes.rst index 4f71748..913bdb4 100644 --- a/doc/source/cql/changes.rst +++ b/doc/source/cql/changes.rst @@ -24,6 +24,7 @@ The following describes the changes in each version of CQL. 3.4.3 ^^^^^ +- Adds a new ``duration `` :ref:`data types ` (:jira:`11873`). - Support for ``GROUP BY`` (:jira:`10707`). - Adds a ``DEFAULT UNSET`` option for ``INSERT JSON`` to ignore omitted columns (:jira:`11424`). - Allows ``null`` as a legal value for TTL on insert and update. It will be treated as equivalent to http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/doc/source/cql/types.rst ---------------------------------------------------------------------- diff --git a/doc/source/cql/types.rst b/doc/source/cql/types.rst index 62e74ec..b0d205b 100644 --- a/doc/source/cql/types.rst +++ b/doc/source/cql/types.rst @@ -47,6 +47,7 @@ The native types supported by CQL are: : | DATE : | DECIMAL : | DOUBLE + : | DURATION : | FLOAT : | INET : | INT @@ -77,6 +78,7 @@ The following table gives additional informations on the native data types, and :token:`float` ``double`` :token:`integer` 64-bit IEEE-754 floating point :token:`float` + ``duration`` :token:`duration`, A duration with nanosecond precision. See :ref:`durations` below for details ``float`` :token:`integer`, 32-bit IEEE-754 floating point :token:`float` ``inet`` :token:`string` An IP address, either IPv4 (4 bytes long) or IPv6 (16 bytes long). Note that @@ -179,6 +181,46 @@ time: - ``'08:12:54.123456'`` - ``'08:12:54.123456789'`` +.. _durations: + +Working with durations +^^^^^^^^^^^^^^^^^^^^^^ + +Values of the ``duration`` type are encoded as 3 signed integer of variable lengths. The first integer represents the +number of months, the second the number of days and the third the number of nanoseconds. This is due to the fact that +the number of days in a month can change, and a day can have 23 or 25 hours depending on the daylight saving. + +A duration can be input as: + + #. ``(quantity unit)+`` like ``12h30m`` where the unit can be: + + * ``y``: years (12 months) + * ``mo``: months (1 month) + * ``w``: weeks (7 days) + * ``d``: days (1 day) + * ``h``: hours (3,600,000,000,000 nanoseconds) + * ``m``: minutes (60,000,000,000 nanoseconds) + * ``s``: seconds (1,000,000,000 nanoseconds) + * ``ms``: milliseconds (1,000,000 nanoseconds) + * ``us`` or ``µs`` : microseconds (1000 nanoseconds) + * ``ns``: nanoseconds (1 nanosecond) + #. ISO 8601 format: ``P[n]Y[n]M[n]DT[n]H[n]M[n]S or P[n]W`` + #. ISO 8601 alternative format: ``P[YYYY]-[MM]-[DD]T[hh]:[mm]:[ss]`` + +For example:: + + INSERT INTO RiderResults (rider, race, result) VALUES ('Christopher Froome', 'Tour de France', 89h4m48s); + INSERT INTO RiderResults (rider, race, result) VALUES ('BARDET Romain', 'Tour de France', PT89H8M53S); + INSERT INTO RiderResults (rider, race, result) VALUES ('QUINTANA Nairo', 'Tour de France', P0000-00-00T89:09:09); + +.. _duration-limitation: + +Duration columns cannot be used in a table's ``PRIMARY KEY``. This limitation is due to the fact that +durations cannot be ordered. It is effectively not possible to know if ``1mo`` is greater than ``29d`` without a date +context. + +A ``1d`` duration is not equals to a ``24h`` one as the duration type has been created to be able to support daylight +saving. .. _collections: http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/pylib/cqlshlib/cql3handling.py ---------------------------------------------------------------------- diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py index f388f4c..d67175f 100644 --- a/pylib/cqlshlib/cql3handling.py +++ b/pylib/cqlshlib/cql3handling.py @@ -18,8 +18,8 @@ from .cqlhandling import CqlParsingRuleSet, Hint from cassandra.metadata import maybe_escape_name -simple_cql_types = set(('ascii', 'bigint', 'blob', 'boolean', 'counter', 'date', 'decimal', 'double', 'float', 'inet', 'int', - 'smallint', 'text', 'time', 'timestamp', 'timeuuid', 'tinyint', 'uuid', 'varchar', 'varint')) +simple_cql_types = set(('ascii', 'bigint', 'blob', 'boolean', 'counter', 'date', 'decimal', 'double', 'duration', 'float', + 'inet', 'int', 'smallint', 'text', 'time', 'timestamp', 'timeuuid', 'tinyint', 'uuid', 'varchar', 'varint')) simple_cql_types.difference_update(('set', 'map', 'list')) from . import helptopics http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/pylib/cqlshlib/displaying.py ---------------------------------------------------------------------- diff --git a/pylib/cqlshlib/displaying.py b/pylib/cqlshlib/displaying.py index f3b7a64..889d814 100644 --- a/pylib/cqlshlib/displaying.py +++ b/pylib/cqlshlib/displaying.py @@ -113,6 +113,7 @@ DEFAULT_VALUE_COLORS = dict( inet=GREEN, boolean=GREEN, uuid=GREEN, + duration=GREEN, collection=BLUE, reset=ANSI_RESET, ) http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/pylib/cqlshlib/formatting.py ---------------------------------------------------------------------- diff --git a/pylib/cqlshlib/formatting.py b/pylib/cqlshlib/formatting.py index 5364c18..daa6529 100644 --- a/pylib/cqlshlib/formatting.py +++ b/pylib/cqlshlib/formatting.py @@ -21,6 +21,7 @@ import math import os import re import sys +import six import platform import wcwidth @@ -328,6 +329,7 @@ formatter_for('long')(format_integer_type) formatter_for('int')(format_integer_type) formatter_for('bigint')(format_integer_type) formatter_for('varint')(format_integer_type) +formatter_for('duration')(format_integer_type) @formatter_for('datetime') @@ -384,6 +386,79 @@ def format_value_time(val, colormap, **_): return format_python_formatted_type(val, colormap, 'time') +@formatter_for('Duration') +def format_value_duration(val, colormap, **_): + buf = six.iterbytes(val) + months = decode_vint(buf) + days = decode_vint(buf) + nanoseconds = decode_vint(buf) + return format_python_formatted_type(duration_as_str(months, days, nanoseconds), colormap, 'duration') + + +def duration_as_str(months, days, nanoseconds): + builder = list() + if months < 0 or days < 0 or nanoseconds < 0: + builder.append('-') + + remainder = append(builder, abs(months), MONTHS_PER_YEAR, "y") + append(builder, remainder, 1, "mo") + append(builder, abs(days), 1, "d") + + if nanoseconds != 0: + remainder = append(builder, abs(nanoseconds), NANOS_PER_HOUR, "h") + remainder = append(builder, remainder, NANOS_PER_MINUTE, "m") + remainder = append(builder, remainder, NANOS_PER_SECOND, "s") + remainder = append(builder, remainder, NANOS_PER_MILLI, "ms") + remainder = append(builder, remainder, NANOS_PER_MICRO, "us") + append(builder, remainder, 1, "ns") + + return ''.join(builder) + + +def append(builder, dividend, divisor, unit): + if dividend == 0 or dividend < divisor: + return dividend + + builder.append(str(dividend / divisor)) + builder.append(unit) + return dividend % divisor + + +def decode_vint(buf): + return decode_zig_zag_64(decode_unsigned_vint(buf)) + + +def decode_unsigned_vint(buf): + """ + Cassandra vints are encoded differently than the varints used in protocol buffer. + The Cassandra vints are encoded with the most significant group first. The most significant byte will contains + the information about how many extra bytes need to be read as well as the most significant bits of the integer. + The number extra bytes to read is encoded as 1 bits on the left side. + For example, if we need to read 3 more bytes the first byte will start with 1110. + """ + + first_byte = buf.next() + if (first_byte >> 7) == 0: + return first_byte + + size = number_of_extra_bytes_to_read(first_byte) + retval = first_byte & (0xff >> size) + for i in range(size): + b = buf.next() + retval <<= 8 + retval |= b & 0xff + + return retval + + +def number_of_extra_bytes_to_read(b): + return 8 - (~b & 0xff).bit_length() + + +def decode_zig_zag_64(n): + return (n >> 1) ^ -(n & 1) + + @formatter_for('str') def format_value_text(val, encoding, colormap, quote=False, **_): escapedval = val.replace(u'\\', u'\\\\') @@ -501,3 +576,10 @@ def format_value_utype(val, cqltype, encoding, colormap, date_time_format, float + rb displaywidth = 4 * len(subs) + sum(k.displaywidth + v.displaywidth for (k, v) in subs) return FormattedValue(bval, coloredval, displaywidth) + +NANOS_PER_MICRO = 1000 +NANOS_PER_MILLI = 1000 * NANOS_PER_MICRO +NANOS_PER_SECOND = 1000 * NANOS_PER_MILLI +NANOS_PER_MINUTE = 60 * NANOS_PER_SECOND +NANOS_PER_HOUR = 60 * NANOS_PER_MINUTE +MONTHS_PER_YEAR = 12 http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/antlr/Lexer.g ---------------------------------------------------------------------- diff --git a/src/antlr/Lexer.g b/src/antlr/Lexer.g index cbbfd6d..23cbed6 100644 --- a/src/antlr/Lexer.g +++ b/src/antlr/Lexer.g @@ -154,6 +154,7 @@ K_BOOLEAN: B O O L E A N; K_COUNTER: C O U N T E R; K_DECIMAL: D E C I M A L; K_DOUBLE: D O U B L E; +K_DURATION: D U R A T I O N; K_FLOAT: F L O A T; K_INET: I N E T; K_INT: I N T; @@ -275,6 +276,20 @@ fragment EXPONENT : E ('+' | '-')? DIGIT+ ; +fragment DURATION_UNIT + : Y + | M O + | W + | D + | H + | M + | S + | M S + | U S + | '\u00B5' S + | N S + ; + INTEGER : '-'? DIGIT+ ; @@ -299,6 +314,13 @@ BOOLEAN : T R U E | F A L S E ; +DURATION + : '-'? DIGIT+ DURATION_UNIT (DIGIT+ DURATION_UNIT)* + | '-'? 'P' (DIGIT+ 'Y')? (DIGIT+ 'M')? (DIGIT+ 'D')? ('T' (DIGIT+ 'H')? (DIGIT+ 'M')? (DIGIT+ 'S')?)? // ISO 8601 "format with designators" + | '-'? 'P' DIGIT+ 'W' + | '-'? 'P' DIGIT DIGIT DIGIT DIGIT '-' DIGIT DIGIT '-' DIGIT DIGIT 'T' DIGIT DIGIT ':' DIGIT DIGIT ':' DIGIT DIGIT // ISO 8601 "alternative format" + ; + IDENT : LETTER (LETTER | DIGIT | '_')* ; http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/antlr/Parser.g ---------------------------------------------------------------------- diff --git a/src/antlr/Parser.g b/src/antlr/Parser.g index 7d3c93b..3d06dc3 100644 --- a/src/antlr/Parser.g +++ b/src/antlr/Parser.g @@ -1255,6 +1255,7 @@ constant returns [Constants.Literal constant] | t=INTEGER { $constant = Constants.Literal.integer($t.text); } | t=FLOAT { $constant = Constants.Literal.floatingPoint($t.text); } | t=BOOLEAN { $constant = Constants.Literal.bool($t.text); } + | t=DURATION { $constant = Constants.Literal.duration($t.text);} | t=UUID { $constant = Constants.Literal.uuid($t.text); } | t=HEXNUMBER { $constant = Constants.Literal.hex($t.text); } | { String sign=""; } ('-' {sign = "-"; } )? t=(K_NAN | K_INFINITY) { $constant = Constants.Literal.floatingPoint(sign + $t.text); } @@ -1556,6 +1557,7 @@ native_type returns [CQL3Type t] | K_COUNTER { $t = CQL3Type.Native.COUNTER; } | K_DECIMAL { $t = CQL3Type.Native.DECIMAL; } | K_DOUBLE { $t = CQL3Type.Native.DOUBLE; } + | K_DURATION { $t = CQL3Type.Native.DURATION; } | K_FLOAT { $t = CQL3Type.Native.FLOAT; } | K_INET { $t = CQL3Type.Native.INET;} | K_INT { $t = CQL3Type.Native.INT; } http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/java/org/apache/cassandra/cql3/CQL3Type.java ---------------------------------------------------------------------- diff --git a/src/java/org/apache/cassandra/cql3/CQL3Type.java b/src/java/org/apache/cassandra/cql3/CQL3Type.java index cf7e18a..94b8f6d 100644 --- a/src/java/org/apache/cassandra/cql3/CQL3Type.java +++ b/src/java/org/apache/cassandra/cql3/CQL3Type.java @@ -26,6 +26,7 @@ import org.slf4j.LoggerFactory; import org.apache.cassandra.config.Schema; import org.apache.cassandra.db.marshal.*; +import org.apache.cassandra.db.marshal.CollectionType.Kind; import org.apache.cassandra.exceptions.InvalidRequestException; import org.apache.cassandra.exceptions.ConfigurationException; import org.apache.cassandra.exceptions.SyntaxException; @@ -70,6 +71,7 @@ public interface CQL3Type DATE (SimpleDateType.instance), DECIMAL (DecimalType.instance), DOUBLE (DoubleType.instance), + DURATION (DurationType.instance), EMPTY (EmptyType.instance), FLOAT (FloatType.instance), INET (InetAddressType.instance), @@ -494,6 +496,11 @@ public interface CQL3Type return true; } + public boolean isDuration() + { + return false; + } + public boolean isCounter() { return false; @@ -595,6 +602,11 @@ public interface CQL3Type return type == Native.COUNTER; } + public boolean isDuration() + { + return type == Native.DURATION; + } + @Override public String toString() { @@ -656,10 +668,15 @@ public interface CQL3Type if (values.isCounter() && !isInternal) throw new InvalidRequestException("Counters are not allowed inside collections: " + this); + if (values.isDuration() && kind == Kind.SET) + throw new InvalidRequestException("Durations are not allowed inside sets: " + this); + if (keys != null) { if (keys.isCounter()) throw new InvalidRequestException("Counters are not allowed inside collections: " + this); + if (keys.isDuration()) + throw new InvalidRequestException("Durations are not allowed as map keys: " + this); if (!frozen && keys.supportsFreezing() && !keys.frozen) throwNestedNonFrozenError(keys); } http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/java/org/apache/cassandra/cql3/Constants.java ---------------------------------------------------------------------- diff --git a/src/java/org/apache/cassandra/cql3/Constants.java b/src/java/org/apache/cassandra/cql3/Constants.java index f108e8b..c701b71 100644 --- a/src/java/org/apache/cassandra/cql3/Constants.java +++ b/src/java/org/apache/cassandra/cql3/Constants.java @@ -37,7 +37,7 @@ public abstract class Constants public enum Type { - STRING, INTEGER, UUID, FLOAT, BOOLEAN, HEX; + STRING, INTEGER, UUID, FLOAT, BOOLEAN, HEX, DURATION; } private static class UnsetLiteral extends Term.Raw @@ -156,6 +156,11 @@ public abstract class Constants return new Literal(Type.HEX, text); } + public static Literal duration(String text) + { + return new Literal(Type.DURATION, text); + } + public Value prepare(String keyspace, ColumnSpecification receiver) throws InvalidRequestException { if (!testAssignment(keyspace, receiver).isAssignable()) @@ -221,6 +226,7 @@ public abstract class Constants case DATE: case DECIMAL: case DOUBLE: + case DURATION: case FLOAT: case INT: case SMALLINT: @@ -262,6 +268,13 @@ public abstract class Constants return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE; } break; + case DURATION: + switch (nt) + { + case DURATION: + return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE; + } + break; } return AssignmentTestable.TestResult.NOT_ASSIGNABLE; } http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/java/org/apache/cassandra/cql3/Duration.java ---------------------------------------------------------------------- diff --git a/src/java/org/apache/cassandra/cql3/Duration.java b/src/java/org/apache/cassandra/cql3/Duration.java new file mode 100644 index 0000000..48f8850 --- /dev/null +++ b/src/java/org/apache/cassandra/cql3/Duration.java @@ -0,0 +1,590 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.cql3; + +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +import com.google.common.base.Objects; + +import org.apache.cassandra.serializers.MarshalException; + +import static org.apache.cassandra.cql3.statements.RequestValidations.checkTrue; +import static org.apache.cassandra.cql3.statements.RequestValidations.invalidRequest; + +/** + * Represents a duration. A durations store separately months, days, and seconds due to the fact that + * the number of days in a month varies, and a day can have 23 or 25 hours if a daylight saving is involved. + */ +public final class Duration +{ + public static final long NANOS_PER_MICRO = 1000L; + public static final long NANOS_PER_MILLI = 1000 * NANOS_PER_MICRO; + public static final long NANOS_PER_SECOND = 1000 * NANOS_PER_MILLI; + public static final long NANOS_PER_MINUTE = 60 * NANOS_PER_SECOND; + public static final long NANOS_PER_HOUR = 60 * NANOS_PER_MINUTE; + public static final int DAYS_PER_WEEK = 7; + public static final int MONTHS_PER_YEAR = 12; + + /** + * The Regexp used to parse the duration provided as String. + */ + private static final Pattern STANDARD_PATTERN = + Pattern.compile("\\G(\\d+)(y|Y|mo|MO|mO|Mo|w|W|d|D|h|H|s|S|ms|MS|mS|Ms|us|US|uS|Us|µs|µS|ns|NS|nS|Ns|m|M)"); + + /** + * The Regexp used to parse the duration when provided in the ISO 8601 format with designators. + */ + private static final Pattern ISO8601_PATTERN = + Pattern.compile("P((\\d+)Y)?((\\d+)M)?((\\d+)D)?(T((\\d+)H)?((\\d+)M)?((\\d+)S)?)?"); + + /** + * The Regexp used to parse the duration when provided in the ISO 8601 format with designators. + */ + private static final Pattern ISO8601_WEEK_PATTERN = Pattern.compile("P(\\d+)W"); + + /** + * The Regexp used to parse the duration when provided in the ISO 8601 alternative format. + */ + private static final Pattern ISO8601_ALTERNATIVE_PATTERN = + Pattern.compile("P(\\d{4})-(\\d{2})-(\\d{2})T(\\d{2}):(\\d{2}):(\\d{2})"); + + /** + * The number of months. + */ + private final int months; + + /** + * The number of days. + */ + private final int days; + + /** + * The number of nanoseconds. + */ + private final long nanoseconds; + + /** + * Creates a duration. A duration can be negative. + * In this case all the non zero values must be negatives. + * + * @param months the number of months + * @param days the number of days + * @param nanoseconds the number of nanoseconds + */ + private Duration(int months, int days, long nanoseconds) + { + // Makes sure that all the values are negatives if one of them is + assert (months >= 0 && days >= 0 && nanoseconds >= 0) + || ((months <= 0 && days <=0 && nanoseconds <=0)); + + this.months = months; + this.days = days; + this.nanoseconds = nanoseconds; + } + + public static Duration newInstance(int months, int days, long nanoseconds) + { + return new Duration(months, days, nanoseconds); + } + + /** + * Converts a String into a duration. + *

The accepted formats are: + *

    + *
  • multiple digits followed by a time unit like: 12h30m where the time unit can be: + *
      + *
    • {@code y}: years
      • + *
    • {@code m}: months
      • + *
    • {@code w}: weeks
      • + *
    • {@code d}: days
      • + *
    • {@code h}: hours
      • + *
    • {@code m}: minutes
      • + *
    • {@code s}: seconds
      • + *
    • {@code ms}: milliseconds
      • + *
    • {@code us} or {@code µs}: microseconds
      • + *
    • {@code ns}: nanoseconds
      • + *
    + *
    • + *
  • ISO 8601 format: P[n]Y[n]M[n]DT[n]H[n]M[n]S or P[n]W
    • + *
  • ISO 8601 alternative format: P[YYYY]-[MM]-[DD]T[hh]:[mm]:[ss]
    • + *
+ * + * @param input the String to convert + * @return a number of nanoseconds + */ + public static Duration from(String input) + { + boolean isNegative = input.startsWith("-"); + String source = isNegative ? input.substring(1) : input; + + if (source.startsWith("P")) + { + if (source.endsWith("W")) + return parseIso8601WeekFormat(isNegative, source); + + if (source.contains("-")) + return parseIso8601AlternativeFormat(isNegative, source); + + return parseIso8601Format(isNegative, source); + } + return parseStandardFormat(isNegative, source); + } + + private static Duration parseIso8601Format(boolean isNegative, String source) + { + Matcher matcher = ISO8601_PATTERN.matcher(source); + if (!matcher.matches()) + throw invalidRequest("Unable to convert '%s' to a duration", source); + + Builder builder = new Builder(isNegative); + if (matcher.group(1) != null) + builder.addYears(groupAsLong(matcher, 2)); + + if (matcher.group(3) != null) + builder.addMonths(groupAsLong(matcher, 4)); + + if (matcher.group(5) != null) + builder.addDays(groupAsLong(matcher, 6)); + + // Checks if the String contains time information + if (matcher.group(7) != null) + { + if (matcher.group(8) != null) + builder.addHours(groupAsLong(matcher, 9)); + + if (matcher.group(10) != null) + builder.addMinutes(groupAsLong(matcher, 11)); + + if (matcher.group(12) != null) + builder.addSeconds(groupAsLong(matcher, 13)); + } + return builder.build(); + } + + private static Duration parseIso8601AlternativeFormat(boolean isNegative, String source) + { + Matcher matcher = ISO8601_ALTERNATIVE_PATTERN.matcher(source); + if (!matcher.matches()) + throw invalidRequest("Unable to convert '%s' to a duration", source); + + return new Builder(isNegative).addYears(groupAsLong(matcher, 1)) + .addMonths(groupAsLong(matcher, 2)) + .addDays(groupAsLong(matcher, 3)) + .addHours(groupAsLong(matcher, 4)) + .addMinutes(groupAsLong(matcher, 5)) + .addSeconds(groupAsLong(matcher, 6)) + .build(); + } + + private static Duration parseIso8601WeekFormat(boolean isNegative, String source) + { + Matcher matcher = ISO8601_WEEK_PATTERN.matcher(source); + if (!matcher.matches()) + throw invalidRequest("Unable to convert '%s' to a duration", source); + + return new Builder(isNegative).addWeeks(groupAsLong(matcher, 1)) + .build(); + } + + private static Duration parseStandardFormat(boolean isNegative, String source) + { + Matcher matcher = STANDARD_PATTERN.matcher(source); + if (!matcher.find()) + throw invalidRequest("Unable to convert '%s' to a duration", source); + + Builder builder = new Builder(isNegative); + boolean done = false; + + do + { + long number = groupAsLong(matcher, 1); + String symbol = matcher.group(2); + add(builder, number, symbol); + done = matcher.end() == source.length(); + } + while (matcher.find()); + + if (!done) + throw invalidRequest("Unable to convert '%s' to a duration", source); + + return builder.build(); + } + + private static long groupAsLong(Matcher matcher, int group) + { + return Long.parseLong(matcher.group(group)); + } + + private static Builder add(Builder builder, long number, String symbol) + { + switch (symbol.toLowerCase()) + { + case "y": return builder.addYears(number); + case "mo": return builder.addMonths(number); + case "w": return builder.addWeeks(number); + case "d": return builder.addDays(number); + case "h": return builder.addHours(number); + case "m": return builder.addMinutes(number); + case "s": return builder.addSeconds(number); + case "ms": return builder.addMillis(number); + case "us": + case "µs": return builder.addMicros(number); + case "ns": return builder.addNanos(number); + } + throw new MarshalException(String.format("Unknown duration symbol '%s'", symbol)); + } + + public int getMonths() + { + return months; + } + + public int getDays() + { + return days; + } + + public long getNanoseconds() + { + return nanoseconds; + } + + @Override + public int hashCode() + { + return Objects.hashCode(days, months, nanoseconds); + } + + @Override + public boolean equals(Object obj) + { + if (!(obj instanceof Duration)) + return false; + + Duration other = (Duration) obj; + return days == other.days + && months == other.months + && nanoseconds == other.nanoseconds; + } + + @Override + public String toString() + { + StringBuilder builder = new StringBuilder(); + + if (months < 0 || days < 0 || nanoseconds < 0) + builder.append('-'); + + long remainder = append(builder, Math.abs(months), MONTHS_PER_YEAR, "y"); + append(builder, remainder, 1, "mo"); + + append(builder, Math.abs(days), 1, "d"); + + if (nanoseconds != 0) + { + remainder = append(builder, Math.abs(nanoseconds), NANOS_PER_HOUR, "h"); + remainder = append(builder, remainder, NANOS_PER_MINUTE, "m"); + remainder = append(builder, remainder, NANOS_PER_SECOND, "s"); + remainder = append(builder, remainder, NANOS_PER_MILLI, "ms"); + remainder = append(builder, remainder, NANOS_PER_MICRO, "us"); + append(builder, remainder, 1, "ns"); + } + return builder.toString(); + } + + /** + * Appends the result of the division to the specified builder if the dividend is not zero. + * + * @param builder the builder to append to + * @param dividend the dividend + * @param divisor the divisor + * @param unit the time unit to append after the result of the division + * @return the remainder of the division + */ + private static long append(StringBuilder builder, long dividend, long divisor, String unit) + { + if (dividend == 0 || dividend < divisor) + return dividend; + + builder.append(dividend / divisor).append(unit); + return dividend % divisor; + } + + private static class Builder + { + /** + * {@code true} if the duration is a negative one, {@code false} otherwise. + */ + private final boolean isNegative; + + /** + * The number of months. + */ + private int months; + + /** + * The number of days. + */ + private int days; + + /** + * The number of nanoseconds. + */ + private long nanoseconds; + + /** + * We need to make sure that the values for each units are provided in order. + */ + private int currentUnitIndex; + + public Builder(boolean isNegative) + { + this.isNegative = isNegative; + } + + /** + * Adds the specified amount of years. + * + * @param numberOfYears the number of years to add. + * @return this {@code Builder} + */ + public Builder addYears(long numberOfYears) + { + validateOrder(1); + validateMonths(numberOfYears, MONTHS_PER_YEAR); + months += numberOfYears * MONTHS_PER_YEAR; + return this; + } + + /** + * Adds the specified amount of months. + * + * @param numberOfMonths the number of months to add. + * @return this {@code Builder} + */ + public Builder addMonths(long numberOfMonths) + { + validateOrder(2); + validateMonths(numberOfMonths, 1); + months += numberOfMonths; + return this; + } + + /** + * Adds the specified amount of weeks. + * + * @param numberOfWeeks the number of weeks to add. + * @return this {@code Builder} + */ + public Builder addWeeks(long numberOfWeeks) + { + validateOrder(3); + validateDays(numberOfWeeks, DAYS_PER_WEEK); + days += numberOfWeeks * DAYS_PER_WEEK; + return this; + } + + /** + * Adds the specified amount of days. + * + * @param numberOfDays the number of days to add. + * @return this {@code Builder} + */ + public Builder addDays(long numberOfDays) + { + validateOrder(4); + validateDays(numberOfDays, 1); + days += numberOfDays; + return this; + } + + /** + * Adds the specified amount of hours. + * + * @param numberOfHours the number of hours to add. + * @return this {@code Builder} + */ + public Builder addHours(long numberOfHours) + { + validateOrder(5); + validateNanos(numberOfHours, NANOS_PER_HOUR); + nanoseconds += numberOfHours * NANOS_PER_HOUR; + return this; + } + + /** + * Adds the specified amount of minutes. + * + * @param numberOfMinutes the number of minutes to add. + * @return this {@code Builder} + */ + public Builder addMinutes(long numberOfMinutes) + { + validateOrder(6); + validateNanos(numberOfMinutes, NANOS_PER_MINUTE); + nanoseconds += numberOfMinutes * NANOS_PER_MINUTE; + return this; + } + + /** + * Adds the specified amount of seconds. + * + * @param numberOfSeconds the number of seconds to add. + * @return this {@code Builder} + */ + public Builder addSeconds(long numberOfSeconds) + { + validateOrder(7); + validateNanos(numberOfSeconds, NANOS_PER_SECOND); + nanoseconds += numberOfSeconds * NANOS_PER_SECOND; + return this; + } + + /** + * Adds the specified amount of milliseconds. + * + * @param numberOfMillis the number of milliseconds to add. + * @return this {@code Builder} + */ + public Builder addMillis(long numberOfMillis) + { + validateOrder(8); + validateNanos(numberOfMillis, NANOS_PER_MILLI); + nanoseconds += numberOfMillis * NANOS_PER_MILLI; + return this; + } + + /** + * Adds the specified amount of microseconds. + * + * @param numberOfMicros the number of microseconds to add. + * @return this {@code Builder} + */ + public Builder addMicros(long numberOfMicros) + { + validateOrder(9); + validateNanos(numberOfMicros, NANOS_PER_MICRO); + nanoseconds += numberOfMicros * NANOS_PER_MICRO; + return this; + } + + /** + * Adds the specified amount of nanoseconds. + * + * @param numberOfNanos the number of nanoseconds to add. + * @return this {@code Builder} + */ + public Builder addNanos(long numberOfNanos) + { + validateOrder(10); + validateNanos(numberOfNanos, 1); + nanoseconds += numberOfNanos; + return this; + } + + /** + * Validates that the total number of months can be stored. + * @param units the number of units that need to be added + * @param monthsPerUnit the number of days per unit + */ + private void validateMonths(long units, int monthsPerUnit) + { + validate(units, (Integer.MAX_VALUE - months) / monthsPerUnit, "months"); + } + + /** + * Validates that the total number of days can be stored. + * @param units the number of units that need to be added + * @param daysPerUnit the number of days per unit + */ + private void validateDays(long units, int daysPerUnit) + { + validate(units, (Integer.MAX_VALUE - days) / daysPerUnit, "days"); + } + + /** + * Validates that the total number of nanoseconds can be stored. + * @param units the number of units that need to be added + * @param nanosPerUnit the number of nanoseconds per unit + */ + private void validateNanos(long units, long nanosPerUnit) + { + validate(units, (Long.MAX_VALUE - nanoseconds) / nanosPerUnit, "nanoseconds"); + } + + /** + * Validates that the specified amount is less than the limit. + * @param units the number of units to check + * @param limit the limit on the number of units + * @param unitName the unit name + */ + private void validate(long units, long limit, String unitName) + { + checkTrue(units <= limit, + "Invalid duration. The total number of %s must be less or equal to %s", + unitName, + Integer.MAX_VALUE); + } + + /** + * Validates that the duration values are added in the proper order. + * @param unitIndex the unit index (e.g. years=1, months=2, ...) + */ + private void validateOrder(int unitIndex) + { + if (unitIndex == currentUnitIndex) + throw invalidRequest("Invalid duration. The %s are specified multiple times", getUnitName(unitIndex)); + + if (unitIndex <= currentUnitIndex) + throw invalidRequest("Invalid duration. The %s should be after %s", + getUnitName(currentUnitIndex), + getUnitName(unitIndex)); + + currentUnitIndex = unitIndex; + } + + /** + * Returns the name of the unit corresponding to the specified index. + * @param unitIndex the unit index + * @return the name of the unit corresponding to the specified index. + */ + private String getUnitName(int unitIndex) + { + switch (unitIndex) + { + case 1: return "years"; + case 2: return "months"; + case 3: return "weeks"; + case 4: return "days"; + case 5: return "hours"; + case 6: return "minutes"; + case 7: return "seconds"; + case 8: return "milliseconds"; + case 9: return "microseconds"; + case 10: return "nanoseconds"; + default: throw new AssertionError("unknown unit index: " + unitIndex); + } + } + + public Duration build() + { + return isNegative ? new Duration(-months, -days, -nanoseconds) : new Duration(months, days, nanoseconds); + } + } +} http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/java/org/apache/cassandra/cql3/SingleColumnRelation.java ---------------------------------------------------------------------- diff --git a/src/java/org/apache/cassandra/cql3/SingleColumnRelation.java b/src/java/org/apache/cassandra/cql3/SingleColumnRelation.java index 4dbb7da..719ef68 100644 --- a/src/java/org/apache/cassandra/cql3/SingleColumnRelation.java +++ b/src/java/org/apache/cassandra/cql3/SingleColumnRelation.java @@ -28,6 +28,7 @@ import org.apache.cassandra.cql3.restrictions.Restriction; import org.apache.cassandra.cql3.restrictions.SingleColumnRestriction; import org.apache.cassandra.cql3.statements.Bound; import org.apache.cassandra.db.marshal.CollectionType; +import org.apache.cassandra.db.marshal.DurationType; import org.apache.cassandra.db.marshal.ListType; import org.apache.cassandra.db.marshal.MapType; import org.apache.cassandra.exceptions.InvalidRequestException; @@ -197,6 +198,8 @@ public final class SingleColumnRelation extends Relation boolean inclusive) throws InvalidRequestException { ColumnDefinition columnDef = entity.prepare(cfm); + checkFalse(columnDef.type instanceof DurationType, "Slice restriction are not supported on duration columns"); + Term term = toTerm(toReceivers(columnDef), value, cfm.ksName, boundNames); return new SingleColumnRestriction.SliceRestriction(columnDef, bound, inclusive, term); } http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/java/org/apache/cassandra/cql3/statements/CreateTableStatement.java ---------------------------------------------------------------------- diff --git a/src/java/org/apache/cassandra/cql3/statements/CreateTableStatement.java b/src/java/org/apache/cassandra/cql3/statements/CreateTableStatement.java index 90f0cdb..7f8eebc 100644 --- a/src/java/org/apache/cassandra/cql3/statements/CreateTableStatement.java +++ b/src/java/org/apache/cassandra/cql3/statements/CreateTableStatement.java @@ -259,6 +259,8 @@ public class CreateTableStatement extends SchemaAlteringStatement AbstractType t = getTypeAndRemove(stmt.columns, alias); if (t.asCQL3Type().getType() instanceof CounterColumnType) throw new InvalidRequestException(String.format("counter type is not supported for PRIMARY KEY part %s", alias)); + if (t.asCQL3Type().getType().referencesDuration()) + throw new InvalidRequestException(String.format("duration type is not supported for PRIMARY KEY part %s", alias)); if (staticColumns.contains(alias)) throw new InvalidRequestException(String.format("Static column %s cannot be part of the PRIMARY KEY", alias)); stmt.keyTypes.add(t); @@ -273,6 +275,8 @@ public class CreateTableStatement extends SchemaAlteringStatement AbstractType type = getTypeAndRemove(stmt.columns, t); if (type.asCQL3Type().getType() instanceof CounterColumnType) throw new InvalidRequestException(String.format("counter type is not supported for PRIMARY KEY part %s", t)); + if (type.asCQL3Type().getType().referencesDuration()) + throw new InvalidRequestException(String.format("duration type is not supported for PRIMARY KEY part %s", t)); if (staticColumns.contains(t)) throw new InvalidRequestException(String.format("Static column %s cannot be part of the PRIMARY KEY", t)); stmt.clusteringTypes.add(type); http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/java/org/apache/cassandra/cql3/statements/CreateViewStatement.java ---------------------------------------------------------------------- diff --git a/src/java/org/apache/cassandra/cql3/statements/CreateViewStatement.java b/src/java/org/apache/cassandra/cql3/statements/CreateViewStatement.java index 5f2ba71..3781a6e 100644 --- a/src/java/org/apache/cassandra/cql3/statements/CreateViewStatement.java +++ b/src/java/org/apache/cassandra/cql3/statements/CreateViewStatement.java @@ -19,7 +19,6 @@ package org.apache.cassandra.cql3.statements; import java.util.*; -import java.util.stream.Collectors; import com.google.common.collect.Iterables; import com.google.common.collect.Sets; @@ -34,6 +33,7 @@ import org.apache.cassandra.cql3.restrictions.StatementRestrictions; import org.apache.cassandra.cql3.selection.RawSelector; import org.apache.cassandra.cql3.selection.Selectable; import org.apache.cassandra.db.marshal.AbstractType; +import org.apache.cassandra.db.marshal.DurationType; import org.apache.cassandra.db.marshal.ReversedType; import org.apache.cassandra.db.view.View; import org.apache.cassandra.exceptions.AlreadyExistsException; @@ -184,6 +184,9 @@ public class CreateViewStatement extends SchemaAlteringStatement if (cdef.isStatic()) throw new InvalidRequestException(String.format("Cannot use Static column '%s' in PRIMARY KEY of materialized view", identifier)); + + if (cdef.type instanceof DurationType) + throw new InvalidRequestException(String.format("Cannot use Duration column '%s' in PRIMARY KEY of materialized view", identifier)); } // build the select statement http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/java/org/apache/cassandra/db/marshal/AbstractType.java ---------------------------------------------------------------------- diff --git a/src/java/org/apache/cassandra/db/marshal/AbstractType.java b/src/java/org/apache/cassandra/db/marshal/AbstractType.java index 2b5503b..8cd40cb 100644 --- a/src/java/org/apache/cassandra/db/marshal/AbstractType.java +++ b/src/java/org/apache/cassandra/db/marshal/AbstractType.java @@ -446,6 +446,11 @@ public abstract class AbstractType implements Comparator, Assignm return false; } + public boolean referencesDuration() + { + return false; + } + /** * This must be overriden by subclasses if necessary so that for any * AbstractType, this == TypeParser.parse(toString()). http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/java/org/apache/cassandra/db/marshal/DurationType.java ---------------------------------------------------------------------- diff --git a/src/java/org/apache/cassandra/db/marshal/DurationType.java b/src/java/org/apache/cassandra/db/marshal/DurationType.java new file mode 100644 index 0000000..e6e1415 --- /dev/null +++ b/src/java/org/apache/cassandra/db/marshal/DurationType.java @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.db.marshal; + +import java.nio.ByteBuffer; + +import org.apache.cassandra.cql3.CQL3Type; +import org.apache.cassandra.cql3.Constants; +import org.apache.cassandra.cql3.Duration; +import org.apache.cassandra.cql3.Term; +import org.apache.cassandra.serializers.DurationSerializer; +import org.apache.cassandra.serializers.MarshalException; +import org.apache.cassandra.serializers.TypeSerializer; +import org.apache.cassandra.utils.ByteBufferUtil; + +/** + * Represents a duration. The duration is stored as months, days, and nanoseconds. This is done + *

Internally he duration is stored as months (unsigned integer), days (unsigned integer), and nanoseconds.

+ */ +public class DurationType extends AbstractType +{ + public static final DurationType instance = new DurationType(); + + DurationType() + { + super(ComparisonType.NOT_COMPARABLE); + } // singleton + + public ByteBuffer fromString(String source) throws MarshalException + { + // Return an empty ByteBuffer for an empty string. + if (source.isEmpty()) + return ByteBufferUtil.EMPTY_BYTE_BUFFER; + + return decompose(Duration.from(source)); + } + + @Override + public boolean isValueCompatibleWithInternal(AbstractType otherType) + { + return this == otherType; + } + + public Term fromJSONObject(Object parsed) throws MarshalException + { + try + { + return new Constants.Value(fromString((String) parsed)); + } + catch (ClassCastException exc) + { + throw new MarshalException(String.format("Expected a string representation of a duration, but got a %s: %s", + parsed.getClass().getSimpleName(), parsed)); + } + } + + @Override + public String toJSONString(ByteBuffer buffer, int protocolVersion) + { + return getSerializer().deserialize(buffer).toString(); + } + + @Override + public TypeSerializer getSerializer() + { + return DurationSerializer.instance; + } + + @Override + public CQL3Type asCQL3Type() + { + return CQL3Type.Native.DURATION; + } + + @Override + public boolean referencesDuration() + { + return true; + } +} http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/java/org/apache/cassandra/db/marshal/ListType.java ---------------------------------------------------------------------- diff --git a/src/java/org/apache/cassandra/db/marshal/ListType.java b/src/java/org/apache/cassandra/db/marshal/ListType.java index ed843b1..b2d5005 100644 --- a/src/java/org/apache/cassandra/db/marshal/ListType.java +++ b/src/java/org/apache/cassandra/db/marshal/ListType.java @@ -80,6 +80,12 @@ public class ListType extends CollectionType> return getElementsType().referencesUserType(userTypeName); } + @Override + public boolean referencesDuration() + { + return getElementsType().referencesDuration(); + } + public AbstractType getElementsType() { return elements; http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/java/org/apache/cassandra/db/marshal/MapType.java ---------------------------------------------------------------------- diff --git a/src/java/org/apache/cassandra/db/marshal/MapType.java b/src/java/org/apache/cassandra/db/marshal/MapType.java index d5cf959..542a330 100644 --- a/src/java/org/apache/cassandra/db/marshal/MapType.java +++ b/src/java/org/apache/cassandra/db/marshal/MapType.java @@ -81,6 +81,13 @@ public class MapType extends CollectionType> getValuesType().referencesUserType(userTypeName); } + @Override + public boolean referencesDuration() + { + // Maps cannot be created with duration as keys + return getValuesType().referencesDuration(); + } + public AbstractType getKeysType() { return keys; http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/java/org/apache/cassandra/db/marshal/TupleType.java ---------------------------------------------------------------------- diff --git a/src/java/org/apache/cassandra/db/marshal/TupleType.java b/src/java/org/apache/cassandra/db/marshal/TupleType.java index 24498aa..f5e7867 100644 --- a/src/java/org/apache/cassandra/db/marshal/TupleType.java +++ b/src/java/org/apache/cassandra/db/marshal/TupleType.java @@ -79,6 +79,12 @@ public class TupleType extends AbstractType return allTypes().stream().anyMatch(f -> f.referencesUserType(name)); } + @Override + public boolean referencesDuration() + { + return allTypes().stream().anyMatch(f -> f.referencesDuration()); + } + public AbstractType type(int i) { return types.get(i); http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/java/org/apache/cassandra/db/marshal/UserType.java ---------------------------------------------------------------------- diff --git a/src/java/org/apache/cassandra/db/marshal/UserType.java b/src/java/org/apache/cassandra/db/marshal/UserType.java index a9c02f3..cd181cc 100644 --- a/src/java/org/apache/cassandra/db/marshal/UserType.java +++ b/src/java/org/apache/cassandra/db/marshal/UserType.java @@ -379,6 +379,12 @@ public class UserType extends TupleType } @Override + public boolean referencesDuration() + { + return fieldTypes().stream().anyMatch(f -> f.referencesDuration()); + } + + @Override public String toString() { return this.toString(false); http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/src/java/org/apache/cassandra/serializers/DurationSerializer.java ---------------------------------------------------------------------- diff --git a/src/java/org/apache/cassandra/serializers/DurationSerializer.java b/src/java/org/apache/cassandra/serializers/DurationSerializer.java new file mode 100644 index 0000000..d139b9e --- /dev/null +++ b/src/java/org/apache/cassandra/serializers/DurationSerializer.java @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.serializers; + +import java.io.IOException; +import java.nio.ByteBuffer; + +import org.apache.cassandra.cql3.Duration; +import org.apache.cassandra.io.util.DataInputBuffer; +import org.apache.cassandra.io.util.DataOutputBufferFixed; +import org.apache.cassandra.utils.ByteBufferUtil; +import org.apache.cassandra.utils.vint.VIntCoding; + +public final class DurationSerializer implements TypeSerializer +{ + public static final DurationSerializer instance = new DurationSerializer(); + + public ByteBuffer serialize(Duration duration) + { + if (duration == null) + return ByteBufferUtil.EMPTY_BYTE_BUFFER; + + long months = duration.getMonths(); + long days = duration.getDays(); + long nanoseconds = duration.getNanoseconds(); + + int size = VIntCoding.computeVIntSize(months) + + VIntCoding.computeVIntSize(days) + + VIntCoding.computeVIntSize(nanoseconds); + + try (DataOutputBufferFixed output = new DataOutputBufferFixed(size)) + { + output.writeVInt(months); + output.writeVInt(days); + output.writeVInt(nanoseconds); + return output.buffer(); + } + catch (IOException e) + { + // this should never happen with a DataOutputBufferFixed + throw new AssertionError("Unexpected error", e); + } + } + + public Duration deserialize(ByteBuffer bytes) + { + if (bytes.remaining() == 0) + return null; + + try (DataInputBuffer in = new DataInputBuffer(bytes, true)) + { + int months = (int) in.readVInt(); + int days = (int) in.readVInt(); + long nanoseconds = in.readVInt(); + return Duration.newInstance(months, days, nanoseconds); + } + catch (IOException e) + { + // this should never happen with a DataInputBuffer + throw new AssertionError("Unexpected error", e); + } + } + + public void validate(ByteBuffer bytes) throws MarshalException + { + if (bytes.remaining() < 3) + throw new MarshalException(String.format("Expected at least 3 bytes for a duration (%d)", bytes.remaining())); + } + + public String toString(Duration duration) + { + return duration == null ? "" : duration.toString(); + } + + public Class getType() + { + return Duration.class; + } +} http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/test/unit/org/apache/cassandra/config/CFMetaDataTest.java ---------------------------------------------------------------------- diff --git a/test/unit/org/apache/cassandra/config/CFMetaDataTest.java b/test/unit/org/apache/cassandra/config/CFMetaDataTest.java index 6e2fa50..78b372e 100644 --- a/test/unit/org/apache/cassandra/config/CFMetaDataTest.java +++ b/test/unit/org/apache/cassandra/config/CFMetaDataTest.java @@ -188,17 +188,17 @@ public class CFMetaDataTest } private static Set primitiveTypes = new HashSet(Arrays.asList(new String[] { "ascii", "bigint", "blob", "boolean", "date", - "decimal", "double", "float", "inet", "int", - "smallint", "text", "time", "timestamp", - "timeuuid", "tinyint", "uuid", "varchar", - "varint" })); + "duration", "decimal", "double", "float", + "inet", "int", "smallint", "text", "time", + "timestamp", "timeuuid", "tinyint", "uuid", + "varchar", "varint" })); @Test public void typeCompatibilityTest() throws Throwable { Map> compatibilityMap = new HashMap<>(); compatibilityMap.put("bigint", new HashSet<>(Arrays.asList(new String[] {"timestamp"}))); - compatibilityMap.put("blob", new HashSet<>(Arrays.asList(new String[] {"ascii", "bigint", "boolean", "date", "decimal", "double", + compatibilityMap.put("blob", new HashSet<>(Arrays.asList(new String[] {"ascii", "bigint", "boolean", "date", "decimal", "double", "duration", "float", "inet", "int", "smallint", "text", "time", "timestamp", "timeuuid", "tinyint", "uuid", "varchar", "varint"}))); compatibilityMap.put("date", new HashSet<>(Arrays.asList(new String[] {"int"}))); http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/test/unit/org/apache/cassandra/cql3/CQL3TypeLiteralTest.java ---------------------------------------------------------------------- diff --git a/test/unit/org/apache/cassandra/cql3/CQL3TypeLiteralTest.java b/test/unit/org/apache/cassandra/cql3/CQL3TypeLiteralTest.java index 43dc267..ee4bb35 100644 --- a/test/unit/org/apache/cassandra/cql3/CQL3TypeLiteralTest.java +++ b/test/unit/org/apache/cassandra/cql3/CQL3TypeLiteralTest.java @@ -207,6 +207,13 @@ public class CQL3TypeLiteralTest } addNativeValue("null", CQL3Type.Native.TIME, null); + for (int i = 0; i < 100; i++) + { + Duration duration = Duration.newInstance(Math.abs(randInt()), Math.abs(randInt()), Math.abs(randLong())); + addNativeValue(DurationSerializer.instance.toString(duration), CQL3Type.Native.DURATION, DurationSerializer.instance.serialize(duration)); + } + addNativeValue("null", CQL3Type.Native.DURATION, null); + // (mostly generates timestamp values with surreal values like in year 14273) for (int i = 0; i < 20; i++) { http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/test/unit/org/apache/cassandra/cql3/CQLTester.java ---------------------------------------------------------------------- diff --git a/test/unit/org/apache/cassandra/cql3/CQLTester.java b/test/unit/org/apache/cassandra/cql3/CQLTester.java index 3bb753f..8b4d359 100644 --- a/test/unit/org/apache/cassandra/cql3/CQLTester.java +++ b/test/unit/org/apache/cassandra/cql3/CQLTester.java @@ -1517,6 +1517,9 @@ public abstract class CQLTester if (value instanceof Float) return FloatType.instance; + if (value instanceof Duration) + return DurationType.instance; + if (value instanceof Double) return DoubleType.instance; http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/test/unit/org/apache/cassandra/cql3/DurationTest.java ---------------------------------------------------------------------- diff --git a/test/unit/org/apache/cassandra/cql3/DurationTest.java b/test/unit/org/apache/cassandra/cql3/DurationTest.java new file mode 100644 index 0000000..b8f4400 --- /dev/null +++ b/test/unit/org/apache/cassandra/cql3/DurationTest.java @@ -0,0 +1,114 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.cassandra.cql3; + +import org.junit.Assert; +import org.junit.Test; + +import static org.junit.Assert.assertEquals; + +import org.apache.cassandra.exceptions.InvalidRequestException; + +import static org.apache.cassandra.cql3.Duration.*; +import static org.apache.cassandra.cql3.Duration.NANOS_PER_HOUR; + +public class DurationTest +{ + @Test + public void testFromStringWithStandardPattern() + { + assertEquals(Duration.newInstance(14, 0, 0), Duration.from("1y2mo")); + assertEquals(Duration.newInstance(-14, 0, 0), Duration.from("-1y2mo")); + assertEquals(Duration.newInstance(14, 0, 0), Duration.from("1Y2MO")); + assertEquals(Duration.newInstance(0, 14, 0), Duration.from("2w")); + assertEquals(Duration.newInstance(0, 2, 10 * NANOS_PER_HOUR), Duration.from("2d10h")); + assertEquals(Duration.newInstance(0, 2, 0), Duration.from("2d")); + assertEquals(Duration.newInstance(0, 0, 30 * NANOS_PER_HOUR), Duration.from("30h")); + assertEquals(Duration.newInstance(0, 0, 30 * NANOS_PER_HOUR + 20 * NANOS_PER_MINUTE), Duration.from("30h20m")); + assertEquals(Duration.newInstance(0, 0, 20 * NANOS_PER_MINUTE), Duration.from("20m")); + assertEquals(Duration.newInstance(0, 0, 56 * NANOS_PER_SECOND), Duration.from("56s")); + assertEquals(Duration.newInstance(0, 0, 567 * NANOS_PER_MILLI), Duration.from("567ms")); + assertEquals(Duration.newInstance(0, 0, 1950 * NANOS_PER_MICRO), Duration.from("1950us")); + assertEquals(Duration.newInstance(0, 0, 1950 * NANOS_PER_MICRO), Duration.from("1950µs")); + assertEquals(Duration.newInstance(0, 0, 1950000), Duration.from("1950000ns")); + assertEquals(Duration.newInstance(0, 0, 1950000), Duration.from("1950000NS")); + assertEquals(Duration.newInstance(0, 0, -1950000), Duration.from("-1950000ns")); + assertEquals(Duration.newInstance(15, 0, 130 * NANOS_PER_MINUTE), Duration.from("1y3mo2h10m")); + } + + @Test + public void testFromStringWithIso8601Pattern() + { + assertEquals(Duration.newInstance(12, 2, 0), Duration.from("P1Y2D")); + assertEquals(Duration.newInstance(14, 0, 0), Duration.from("P1Y2M")); + assertEquals(Duration.newInstance(0, 14, 0), Duration.from("P2W")); + assertEquals(Duration.newInstance(12, 0, 2 * NANOS_PER_HOUR), Duration.from("P1YT2H")); + assertEquals(Duration.newInstance(-14, 0, 0), Duration.from("-P1Y2M")); + assertEquals(Duration.newInstance(0, 2, 0), Duration.from("P2D")); + assertEquals(Duration.newInstance(0, 0, 30 * NANOS_PER_HOUR), Duration.from("PT30H")); + assertEquals(Duration.newInstance(0, 0, 30 * NANOS_PER_HOUR + 20 * NANOS_PER_MINUTE), Duration.from("PT30H20M")); + assertEquals(Duration.newInstance(0, 0, 20 * NANOS_PER_MINUTE), Duration.from("PT20M")); + assertEquals(Duration.newInstance(0, 0, 56 * NANOS_PER_SECOND), Duration.from("PT56S")); + assertEquals(Duration.newInstance(15, 0, 130 * NANOS_PER_MINUTE), Duration.from("P1Y3MT2H10M")); + } + + @Test + public void testFromStringWithIso8601AlternativePattern() + { + assertEquals(Duration.newInstance(12, 2, 0), Duration.from("P0001-00-02T00:00:00")); + assertEquals(Duration.newInstance(14, 0, 0), Duration.from("P0001-02-00T00:00:00")); + assertEquals(Duration.newInstance(12, 0, 2 * NANOS_PER_HOUR), Duration.from("P0001-00-00T02:00:00")); + assertEquals(Duration.newInstance(-14, 0, 0), Duration.from("-P0001-02-00T00:00:00")); + assertEquals(Duration.newInstance(0, 2, 0), Duration.from("P0000-00-02T00:00:00")); + assertEquals(Duration.newInstance(0, 0, 30 * NANOS_PER_HOUR), Duration.from("P0000-00-00T30:00:00")); + assertEquals(Duration.newInstance(0, 0, 30 * NANOS_PER_HOUR + 20 * NANOS_PER_MINUTE), Duration.from("P0000-00-00T30:20:00")); + assertEquals(Duration.newInstance(0, 0, 20 * NANOS_PER_MINUTE), Duration.from("P0000-00-00T00:20:00")); + assertEquals(Duration.newInstance(0, 0, 56 * NANOS_PER_SECOND), Duration.from("P0000-00-00T00:00:56")); + assertEquals(Duration.newInstance(15, 0, 130 * NANOS_PER_MINUTE), Duration.from("P0001-03-00T02:10:00")); + } + + @Test + public void testInvalidDurations() + { + assertInvalidDuration(Long.MAX_VALUE + "d", "Invalid duration. The total number of days must be less or equal to 2147483647"); + assertInvalidDuration("2µ", "Unable to convert '2µ' to a duration"); + assertInvalidDuration("-2µ", "Unable to convert '2µ' to a duration"); + assertInvalidDuration("12.5s", "Unable to convert '12.5s' to a duration"); + assertInvalidDuration("2m12.5s", "Unable to convert '2m12.5s' to a duration"); + assertInvalidDuration("2m-12s", "Unable to convert '2m-12s' to a duration"); + assertInvalidDuration("12s3s", "Invalid duration. The seconds are specified multiple times"); + assertInvalidDuration("12s3m", "Invalid duration. The seconds should be after minutes"); + assertInvalidDuration("1Y3M4D", "Invalid duration. The minutes should be after days"); + assertInvalidDuration("P2Y3W", "Unable to convert 'P2Y3W' to a duration"); + assertInvalidDuration("P0002-00-20", "Unable to convert 'P0002-00-20' to a duration"); + } + + public void assertInvalidDuration(String duration, String expectedErrorMessage) + { + try + { + System.out.println(Duration.from(duration)); + Assert.fail(); + } + catch (InvalidRequestException e) + { + assertEquals(expectedErrorMessage, e.getMessage()); + } + } +} http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/test/unit/org/apache/cassandra/cql3/ViewTest.java ---------------------------------------------------------------------- diff --git a/test/unit/org/apache/cassandra/cql3/ViewTest.java b/test/unit/org/apache/cassandra/cql3/ViewTest.java index aa4772f..ccfadef 100644 --- a/test/unit/org/apache/cassandra/cql3/ViewTest.java +++ b/test/unit/org/apache/cassandra/cql3/ViewTest.java @@ -348,6 +348,27 @@ public class ViewTest extends CQLTester } @Test + public void testDurationsTable() throws Throwable + { + createTable("CREATE TABLE %s (" + + "k int PRIMARY KEY, " + + "result duration)"); + + execute("USE " + keyspace()); + executeNet(protocolVersion, "USE " + keyspace()); + + try + { + createView("mv_duration", "CREATE MATERIALIZED VIEW %s AS SELECT * FROM %%s WHERE result IS NOT NULL AND k IS NOT NULL PRIMARY KEY (result,k)"); + Assert.fail("MV on duration should fail"); + } + catch (InvalidQueryException e) + { + Assert.assertEquals("Cannot use Duration column 'result' in PRIMARY KEY of materialized view", e.getMessage()); + } + } + + @Test public void complexTimestampUpdateTestWithFlush() throws Throwable { complexTimestampUpdateTest(true); http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/test/unit/org/apache/cassandra/cql3/validation/entities/CollectionsTest.java ---------------------------------------------------------------------- diff --git a/test/unit/org/apache/cassandra/cql3/validation/entities/CollectionsTest.java b/test/unit/org/apache/cassandra/cql3/validation/entities/CollectionsTest.java index c3fc05a..0c59d6f 100644 --- a/test/unit/org/apache/cassandra/cql3/validation/entities/CollectionsTest.java +++ b/test/unit/org/apache/cassandra/cql3/validation/entities/CollectionsTest.java @@ -17,11 +17,7 @@ */ package org.apache.cassandra.cql3.validation.entities; -import java.util.Map; -import java.util.Random; -import java.util.Set; -import java.util.Arrays; -import java.util.UUID; +import java.util.*; import org.junit.Test; http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java ---------------------------------------------------------------------- diff --git a/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java b/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java index e930a9e..dadbeb0 100644 --- a/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java +++ b/test/unit/org/apache/cassandra/cql3/validation/entities/JsonTest.java @@ -20,6 +20,7 @@ package org.apache.cassandra.cql3.validation.entities; import org.apache.cassandra.config.DatabaseDescriptor; import org.apache.cassandra.cql3.Json; import org.apache.cassandra.cql3.CQLTester; +import org.apache.cassandra.cql3.Duration; import org.apache.cassandra.dht.ByteOrderedPartitioner; import org.apache.cassandra.serializers.SimpleDateSerializer; import org.apache.cassandra.serializers.TimeSerializer; @@ -81,8 +82,8 @@ public class JsonTest extends CQLTester "mapval map," + "frozenmapval frozen>," + "tupleval frozen>," + - "udtval frozen<" + typeName + ">)"); - + "udtval frozen<" + typeName + ">," + + "durationval duration)"); // fromJson() can only be used when the receiver type is known assertInvalidMessage("fromJson() cannot be used in the selection clause", "SELECT fromJson(asciival) FROM %s", 0, 0); @@ -508,6 +509,16 @@ public class JsonTest extends CQLTester row(0, 1, UUID.fromString("6bddc89a-5644-11e4-97fc-56847afe9799"), set("bar", "foo")) ); + // ================ duration ================ + execute("INSERT INTO %s (k, durationval) VALUES (?, fromJson(?))", 0, "\"53us\""); + assertRows(execute("SELECT k, durationval FROM %s WHERE k = ?", 0), row(0, Duration.newInstance(0, 0, 53000L))); + + execute("INSERT INTO %s (k, durationval) VALUES (?, fromJson(?))", 0, "\"P2W\""); + assertRows(execute("SELECT k, durationval FROM %s WHERE k = ?", 0), row(0, Duration.newInstance(0, 14, 0))); + + assertInvalidMessage("Unable to convert 'xyz' to a duration", + "INSERT INTO %s (k, durationval) VALUES (?, fromJson(?))", 0, "\"xyz\""); + // order of fields shouldn't matter execute("INSERT INTO %s (k, udtval) VALUES (?, fromJson(?))", 0, "{\"b\": \"6bddc89a-5644-11e4-97fc-56847afe9799\", \"a\": 1, \"c\": [\"foo\", \"bar\"]}"); assertRows(execute("SELECT k, udtval.a, udtval.b, udtval.c FROM %s WHERE k = ?", 0), @@ -563,7 +574,8 @@ public class JsonTest extends CQLTester "mapval map, " + "frozenmapval frozen>, " + "tupleval frozen>," + - "udtval frozen<" + typeName + ">)"); + "udtval frozen<" + typeName + ">," + + "durationval duration)"); // toJson() can only be used in selections assertInvalidMessage("toJson() may only be used within the selection clause", @@ -761,6 +773,13 @@ public class JsonTest extends CQLTester assertRows(execute("SELECT k, toJson(udtval) FROM %s WHERE k = ?", 0), row(0, "{\"a\": 1, \"b\": \"6bddc89a-5644-11e4-97fc-56847afe9799\", \"c\": null}") ); + + // ================ duration ================ + execute("INSERT INTO %s (k, durationval) VALUES (?, 12µs)", 0); + assertRows(execute("SELECT k, toJson(durationval) FROM %s WHERE k = ?", 0), row(0, "12us")); + + execute("INSERT INTO %s (k, durationval) VALUES (?, P1Y1M2DT10H5M)", 0); + assertRows(execute("SELECT k, toJson(durationval) FROM %s WHERE k = ?", 0), row(0, "1y1mo2d10h5m")); } @Test http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/test/unit/org/apache/cassandra/cql3/validation/operations/BatchTest.java ---------------------------------------------------------------------- diff --git a/test/unit/org/apache/cassandra/cql3/validation/operations/BatchTest.java b/test/unit/org/apache/cassandra/cql3/validation/operations/BatchTest.java index 66226eb..7df5fb4 100644 --- a/test/unit/org/apache/cassandra/cql3/validation/operations/BatchTest.java +++ b/test/unit/org/apache/cassandra/cql3/validation/operations/BatchTest.java @@ -153,10 +153,10 @@ public class BatchTest extends CQLTester execute("INSERT INTO %s (partitionKey, clustering_1, value) VALUES (0, 6, 6)"); execute("BEGIN BATCH " + - "UPDATE %1$s SET value = 7 WHERE partitionKey = 0 AND clustering_1 = 1" + - "UPDATE %1$s SET value = 8 WHERE partitionKey = 0 AND (clustering_1) = (2)" + - "UPDATE %1$s SET value = 10 WHERE partitionKey = 0 AND clustering_1 IN (3, 4)" + - "UPDATE %1$s SET value = 20 WHERE partitionKey = 0 AND (clustering_1) IN ((5), (6))" + + "UPDATE %1$s SET value = 7 WHERE partitionKey = 0 AND clustering_1 = 1;" + + "UPDATE %1$s SET value = 8 WHERE partitionKey = 0 AND (clustering_1) = (2);" + + "UPDATE %1$s SET value = 10 WHERE partitionKey = 0 AND clustering_1 IN (3, 4);" + + "UPDATE %1$s SET value = 20 WHERE partitionKey = 0 AND (clustering_1) IN ((5), (6));" + "APPLY BATCH;"); assertRows(execute("SELECT * FROM %s"), http://git-wip-us.apache.org/repos/asf/cassandra/blob/ecf05b88/test/unit/org/apache/cassandra/cql3/validation/operations/CreateTest.java ---------------------------------------------------------------------- diff --git a/test/unit/org/apache/cassandra/cql3/validation/operations/CreateTest.java b/test/unit/org/apache/cassandra/cql3/validation/operations/CreateTest.java index da0824f..6912f85 100644 --- a/test/unit/org/apache/cassandra/cql3/validation/operations/CreateTest.java +++ b/test/unit/org/apache/cassandra/cql3/validation/operations/CreateTest.java @@ -27,6 +27,7 @@ import org.apache.cassandra.config.CFMetaData; import org.apache.cassandra.config.Schema; import org.apache.cassandra.config.SchemaConstants; import org.apache.cassandra.cql3.CQLTester; +import org.apache.cassandra.cql3.Duration; import org.apache.cassandra.db.Mutation; import org.apache.cassandra.db.partitions.Partition; import org.apache.cassandra.exceptions.ConfigurationException; @@ -40,6 +41,8 @@ import static junit.framework.Assert.assertEquals; import static junit.framework.Assert.assertFalse; import static junit.framework.Assert.assertTrue; import static junit.framework.Assert.fail; +import static org.apache.cassandra.cql3.Duration.*; +import static org.junit.Assert.assertEquals; public class CreateTest extends CQLTester { @@ -85,6 +88,134 @@ public class CreateTest extends CQLTester "INSERT INTO %s (a, b, c) VALUES (?, ?, ?)", "3", (byte) 1, ByteBufferUtil.EMPTY_BYTE_BUFFER); } + @Test + public void testCreateTableWithDurationColumns() throws Throwable + { + assertInvalidMessage("duration type is not supported for PRIMARY KEY part a", + "CREATE TABLE test (a duration PRIMARY KEY, b int);"); + + assertInvalidMessage("duration type is not supported for PRIMARY KEY part b", + "CREATE TABLE test (a text, b duration, c duration, primary key (a, b));"); + + assertInvalidMessage("duration type is not supported for PRIMARY KEY part b", + "CREATE TABLE test (a text, b duration, c duration, primary key (a, b)) with clustering order by (b DESC);"); + + createTable("CREATE TABLE %s (a int, b int, c duration, primary key (a, b));"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 1y2mo)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 2, -1y2mo)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 3, 1Y2MO)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 4, 2w)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 5, 2d10h)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 6, 30h20m)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 7, 20m)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 8, 567ms)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 9, 1950us)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 10, 1950µs)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 11, 1950000NS)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 12, -1950000ns)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 13, 1y3mo2h10m)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 14, -P1Y2M)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 15, P2D)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 16, PT20M)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 17, P2W)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 18, P1Y3MT2H10M)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 19, P0000-00-00T30:20:00)"); + execute("INSERT INTO %s (a, b, c) VALUES (1, 20, P0001-03-00T02:10:00)"); + + assertRows(execute("SELECT * FROM %s"), + row(1, 1, Duration.newInstance(14, 0, 0)), + row(1, 2, Duration.newInstance(-14, 0, 0)), + row(1, 3, Duration.newInstance(14, 0, 0)), + row(1, 4, Duration.newInstance(0, 14, 0)), + row(1, 5, Duration.newInstance(0, 2, 10 * NANOS_PER_HOUR)), + row(1, 6, Duration.newInstance(0, 0, 30 * NANOS_PER_HOUR + 20 * NANOS_PER_MINUTE)), + row(1, 7, Duration.newInstance(0, 0, 20 * NANOS_PER_MINUTE)), + row(1, 8, Duration.newInstance(0, 0, 567 * NANOS_PER_MILLI)), + row(1, 9, Duration.newInstance(0, 0, 1950 * NANOS_PER_MICRO)), + row(1, 10, Duration.newInstance(0, 0, 1950 * NANOS_PER_MICRO)), + row(1, 11, Duration.newInstance(0, 0, 1950000)), + row(1, 12, Duration.newInstance(0, 0, -1950000)), + row(1, 13, Duration.newInstance(15, 0, 130 * NANOS_PER_MINUTE)), + row(1, 14, Duration.newInstance(-14, 0, 0)), + row(1, 15, Duration.newInstance(0, 2, 0)), + row(1, 16, Duration.newInstance(0, 0, 20 * NANOS_PER_MINUTE)), + row(1, 17, Duration.newInstance(0, 14, 0)), + row(1, 18, Duration.newInstance(15, 0, 130 * NANOS_PER_MINUTE)), + row(1, 19, Duration.newInstance(0, 0, 30 * NANOS_PER_HOUR + 20 * NANOS_PER_MINUTE)), + row(1, 20, Duration.newInstance(15, 0, 130 * NANOS_PER_MINUTE))); + + assertInvalidMessage("Slice restriction are not supported on duration columns", + "SELECT * FROM %s WHERE c > 1y ALLOW FILTERING"); + + assertInvalidMessage("Slice restriction are not supported on duration columns", + "SELECT * FROM %s WHERE c <= 1y ALLOW FILTERING"); + + assertInvalidMessage("Expected at least 3 bytes for a duration (1)", + "INSERT INTO %s (a, b, c) VALUES (?, ?, ?)", 2, 1, (byte) 1); + assertInvalidMessage("Expected at least 3 bytes for a duration (0)", + "INSERT INTO %s (a, b, c) VALUES (?, ?, ?)", 2, 1, ByteBufferUtil.EMPTY_BYTE_BUFFER); + assertInvalidMessage("Invalid duration. The total number of days must be less or equal to 2147483647", + "INSERT INTO %s (a, b, c) VALUES (1, 2, " + Long.MAX_VALUE + "d)"); + + // Test with duration column name + createTable("CREATE TABLE %s (a text PRIMARY KEY, duration duration);"); + + // Test duration within Map + assertInvalidMessage("Durations are not allowed as map keys: map", + "CREATE TABLE test(pk int PRIMARY KEY, m map)"); + + createTable("CREATE TABLE %s(pk int PRIMARY KEY, m map)"); + execute("INSERT INTO %s (pk, m) VALUES (1, {'one month' : 1mo, '60 days' : 60d})"); + assertRows(execute("SELECT * FROM %s"), + row(1, map("one month", Duration.from("1mo"), "60 days", Duration.from("60d")))); + + assertInvalidMessage("duration type is not supported for PRIMARY KEY part m", + "CREATE TABLE %s(m frozen> PRIMARY KEY, v int)"); + + assertInvalidMessage("duration type is not supported for PRIMARY KEY part m", + "CREATE TABLE %s(pk int, m frozen>, v int, PRIMARY KEY (pk, m))"); + + // Test duration within Set + assertInvalidMessage("Durations are not allowed inside sets: set", + "CREATE TABLE %s(pk int PRIMARY KEY, s set)"); + + assertInvalidMessage("Durations are not allowed inside sets: frozen>", + "CREATE TABLE %s(s frozen> PRIMARY KEY, v int)"); + + // Test duration within List + createTable("CREATE TABLE %s(pk int PRIMARY KEY, l list)"); + execute("INSERT INTO %s (pk, l) VALUES (1, [1mo, 60d])"); + assertRows(execute("SELECT * FROM %s"), + row(1, list(Duration.from("1mo"), Duration.from("60d")))); + + assertInvalidMessage("duration type is not supported for PRIMARY KEY part l", + "CREATE TABLE %s(l frozen> PRIMARY KEY, v int)"); + + // Test duration within Tuple + createTable("CREATE TABLE %s(pk int PRIMARY KEY, t tuple)"); + execute("INSERT INTO %s (pk, t) VALUES (1, (1, 1mo))"); + assertRows(execute("SELECT * FROM %s"), + row(1, tuple(1, Duration.from("1mo")))); + + assertInvalidMessage("duration type is not supported for PRIMARY KEY part t", + "CREATE TABLE %s(t frozen> PRIMARY KEY, v int)"); + + // Test duration within UDT + String typename = createType("CREATE TYPE %s (a duration)"); + String myType = KEYSPACE + '.' + typename; + createTable("CREATE TABLE %s(pk int PRIMARY KEY, u " + myType + ")"); + execute("INSERT INTO %s (pk, u) VALUES (1, {a : 1mo})"); + assertRows(execute("SELECT * FROM %s"), + row(1, userType("a", Duration.from("1mo")))); + + assertInvalidMessage("duration type is not supported for PRIMARY KEY part u", + "CREATE TABLE %s(pk int, u frozen<" + myType + ">, v int, PRIMARY KEY(pk, u))"); + + // Test duration with several level of depth + assertInvalidMessage("duration type is not supported for PRIMARY KEY part m", + "CREATE TABLE %s(pk int, m frozen>>>, v int, PRIMARY KEY (pk, m))"); + } + /** * Creation and basic operations on a static table, * migrated from cql_tests.py:TestCQL.static_cf_test()