+ +

+ + +

+ +

The SQL++ Query Language

+ +

1. Introduction
2. Expressions + +
- Operator expressions + +
  - Arithmetic operators
  - Collection operators
  - Comparison operators
  - Logical operators
- Case expressions
- Quantified expressions
- Path expressions
- Primary expressions + +
  - Literals
  - Variable references
  - Parenthesized expressions
  - Function call expressions
  - Constructors
3. Queries + +
- SELECT statements
- SELECT clauses + +
  - Select element/value/raw
  - SQL-style select
  - Select *
  - Select distinct
  - Unnamed projections
  - Abbreviatory field access expressions
- UNNEST clauses + +
  - Inner unnests
  - Left outer unnests
  - Expressing joins using unnests
- FROM clauses + +
  - Binding expressions
  - Multiple from terms
  - Expressing joins using from terms
  - Implicit binding variables
- JOIN clauses + +
  - Inner joins
  - Left outer joins
- GROUP BY clauses + +
  - Group variables
  - Implicit group key variables
  - Implicit group variables
  - Aggregation functions
  - SQL-92 aggregation functions
  - SQL-92 compliant GROUP BY aggregations
  - Column aliases
- WHERE clauases and HAVING clauses
- ORDER BY clauses
- LIMIT clauses
- WITH clauses
- LET clauses
- UNION ALL
- SQL++ Vs. SQL-92
4. Errors + +
- Syntax errors
- Identifier resolution errors
- Type errors
- Resource errors
5. DDL and DML statements + +
- Declarations
- Lifecycle management statements + +
  - Dataverses
  - Datasets
  - Types
  - Functions
- Modification statements + +
  - Inserts
  - Upserts
  - Deletes
Appendix 1. Reserved keywords

+ +

1. Introduction

This document is intended as a reference guide to the full syntax and semantics of the SQL++ Query Language, a SQL-inspired language for working with semistructured data. SQL++ has much in common with SQL, but some differences do exist due to the different data models that the two languages were designed to serve. SQL was designed in the 1970’s for interacting with the flat, schema-ified world of relational databases, while SQL++ is much newer and targets the nested, schema-optional (or even schema-less) world of modern NoSQL systems.

In the context of Apache AsterixDB, SQL++ is intended for working with the Asterix Data Model (ADM),a data model based on a superset of JSON with an enriched and flexible type system. New AsterixDB users are encouraged to read and work through the (much friendlier) guide “AsterixDB 101: An ADM and SQL++ Primer” before attempting to make use of this document. In addition, readers are advised to read through the Asterix Data Model (ADM) reference guide first as well, as an understanding of the data model is a prerequisite to understanding SQL++.

In what follows, we detail the features of the SQL++ language in a grammar-guided manner. We list and briefly explain each of the productions in the SQL++ grammar, offering examples (and results) for clarity.

+ +

2. Expressions

+ +

Expression ::= OperatorExpression | CaseExpression | QuantifiedExpression
+

SQL++ is a highly composable expression language. Each SQL++ expression returns zero or more data model instances. There are three major kinds of expressions in SQL++. At the topmost level, a SQL++ expression can be an OperatorExpression (similar to a mathematical expression), an ConditionalExpression (to choose between alternative values), or a QuantifiedExpression (which yields a boolean value). Each will be detailed as we explore the full SQL++ grammar.

Note that in the following text, words enclosed in angle brackets denote keywords that are not case-sensitive.

Operator expressions

Operators perform a specific operation on the input values or expressions. The syntax of an operator expression is as follows:

+ +

OperatorExpression ::= PathExpression
+                       | Operator OperatorExpression
+                       | OperatorExpression Operator (OperatorExpression)?
+                       | OperatorExpression <BETWEEN> OperatorExpression <AND> OperatorExpression
+

SQL++ provides a full set of operators that you can use within its statements. Here are the categories of operators:

+ +

Arithmetic operators, to perform basic mathematical operations;
Collection operators, to evaluate expressions on collections or objects;
Comparison operators, to compare two expressions;
Logical Operators, to combine operators using Boolean logic.

The following table summarizes the precedence order (from higher to lower) of the major unary and binary operators:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Operator	Operation
EXISTS, NOT EXISTS	collection emptiness testing
^	exponentiation
*, /, %	multiplication, division, modulo
+, -	addition, subtraction
\|\|	string concatenation
IS NULL, IS NOT NULL, IS MISSING, IS NOT MISSING, IS UNKNOWN, IS NOT UNKNOWN	unknown value comparison
BETWEEN, NOT BETWEEN	range comparison (inclusive on both sides)
=, !=, <, >, <=, >=, LIKE, NOT LIKE, IN, NOT IN	comparison
NOT	logical negation
AND	conjunction
OR	disjunction

In general, if any operand evaluates to a MISSING value, the enclosing operator will return MISSING; if none of operands evaluates to a MISSING value but there is an operand evaluates to a NULL value, the encolosing operator will return NULL. However, there are a few exceptions listed in comparison operators and logical operators.

Arithmetic operators

Arithemtic operators are used to exponentiate, add, subtract, multiply, and divide numeric values, or concatenate string values.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Operator	Purpose	Example
+, -	As unary operators, they denote a positive or negative expression	SELECT VALUE -1;
+, -	As binary operators, they add or subtract	SELECT VALUE 1 + 2;
*, /	Multiply, divide	SELECT VALUE 4 / 2.0;
^	Exponentiation	SELECT VALUE 2^3;
\|\|	String concatenation	SELECT VALUE “ab”\|\|“c”\|\|“d”;

Collection operators

Collection operators are used for membership tests (IN, NOT IN) or empty collection tests (EXISTS, NOT EXISTS).

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Operator	Purpose	Example
IN	Membership test	SELECT * FROM ChirpMessages cm WHERE cm.user.lang IN [“en”, “de”];
NOT IN	Non-membership test	SELECT * FROM ChirpMessages cm WHERE cm.user.lang NOT IN [“en”];
EXISTS	Check whether a collection is not empty	SELECT * FROM ChirpMessages cm WHERE EXISTS cm.referredTopics;
NOT EXISTS	Check whether a collection is empty	SELECT * FROM ChirpMessages cm WHERE NOT EXISTS cm.referredTopics;

Comparison operators

Comparison operators are used to compare values. The comparison operators fall into one of two sub-categories: missing value comparisons and regular value comparisons. SQL++ (and JSON) has two ways of representing missing information in a object - the presence of the field with a NULL for its value (as in SQL), and the absence of the field (which JSON permits). For example, the first of the following objects represents Jack, whose friend is Jill. In the other examples, Jake is friendless a la SQL, with a friend field that is NULL, while Joe is friendless in a more natural (for JSON) way, i.e., by not having a friend field.

Examples

{“name”: “Jack”, “friend”: “Jill”}

{“name”: “Jake”, “friend”: NULL}

{“name”: “Joe”}

The following table enumerates all of SQL++’s comparison operators.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Operator	Purpose	Example
IS NULL	Test if a value is NULL	SELECT * FROM ChirpMessages cm WHERE cm.user.name IS NULL;
IS NOT NULL	Test if a value is not NULL	SELECT * FROM ChirpMessages cm WHERE cm.user.name IS NOT NULL;
IS MISSING	Test if a value is MISSING	SELECT * FROM ChirpMessages cm WHERE cm.user.name IS MISSING;
IS NOT MISSING	Test if a value is not MISSING	SELECT * FROM ChirpMessages cm WHERE cm.user.name IS NOT MISSING;
IS UNKNOWN	Test if a value is NULL or MISSING	SELECT * FROM ChirpMessages cm WHERE cm.user.name IS UNKNOWN;
IS NOT UNKNOWN	Test if a value is neither NULL nor MISSING	SELECT * FROM ChirpMessages cm WHERE cm.user.name IS NOT UNKNOWN;
BETWEEN	Test if a value is between a start value and a end value. The comparison is inclusive to both start and end values.	SELECT * FROM ChirpMessages cm WHERE cm.chirpId BETWEEN 10 AND 20;
=	Equality test	SELECT * FROM ChirpMessages cm WHERE cm.chirpId=10;
!=	Inequality test	SELECT * FROM ChirpMessages cm WHERE cm.chirpId!=10;
<	Less than	SELECT * FROM ChirpMessages cm WHERE cm.chirpId<10;
>	Greater than	SELECT * FROM ChirpMessages cm WHERE cm.chirpId>10;
<=	Less than or equal to	SELECT * FROM ChirpMessages cm WHERE cm.chirpId<=10;
>=	Greater than or equal to	SELECT * FROM ChirpMessages cm WHERE cm.chirpId>=10;
LIKE	Test if the left side matches a pattern defined on the right side; in the pattern, “%” matches any string while “_” matches any character.	SELECT * FROM ChirpMessages cm WHERE cm.user.name LIKE “%Giesen%”;
NOT LIKE	Test if the left side does not match a pattern defined on the right side; in the pattern, “%” matches any string while “_” matches any character.	SELECT * FROM ChirpMessages cm WHERE cm.user.name NOT LIKE “%Giesen%”;

The following table summarizes how the missing value comparison operators work.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Operator	Non-NULL/Non-MISSING value	NULL	MISSING
IS NULL	FALSE	TRUE	MISSING
IS NOT NULL	TRUE	FALSE	MISSING
IS MISSING	FALSE	FALSE	TRUE
IS NOT MISSING	TRUE	TRUE	FALSE
IS UNKNOWN	FALSE	TRUE	TRUE
IS NOT UNKNOWN	TRUE	FALSE	FALSE

Logical operators

Logical operators perform logical NOT, AND, and OR operations over Boolean values (TRUE and FALSE) plus NULL and MISSING.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Operator	Purpose	Example
NOT	Returns true if the following condition is false, otherwise returns false	SELECT VALUE NOT TRUE;
AND	Returns true if both branches are true, otherwise returns false	SELECT VALUE TRUE AND FALSE;
OR	Returns true if one branch is true, otherwise returns false	SELECT VALUE FALSE OR FALSE;

The following table is the truth table for AND and OR.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

A	B	A AND B	A OR B
TRUE	TRUE	TRUE	TRUE
TRUE	FALSE	FALSE	TRUE
TRUE	NULL	NULL	TRUE
TRUE	MISSING	MISSING	TRUE
FALSE	FALSE	FALSE	FALSE
FALSE	NULL	FALSE	NULL
FALSE	MISSING	FALSE	MISSING
NULL	NULL	NULL	NULL
NULL	MISSING	MISSING	NULL
MISSING	MISSING	MISSING	MISSING

The following table demonstrates the results of NOT on all possible inputs.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

A	NOT A
TRUE	FALSE
FALSE	TRUE
NULL	NULL
MISSING	MISSING

Case expressions

+ +

CaseExpression ::= SimpleCaseExpression | SearchedCaseExpression
+SimpleCaseExpression ::= <CASE> Expression ( <WHEN> Expression <THEN> Expression )+ ( <ELSE> Expression )? <END>
+SearchedCaseExpression ::= <CASE> ( <WHEN> Expression <THEN> Expression )+ ( <ELSE> Expression )? <END>
+

In a simple CASE expression, the query evaluator searches for the first WHEN … THEN pair in which the WHEN expression is equal to the expression following CASE and returns the expression following THEN. If none of the WHEN … THEN pairs meet this condition, and an ELSE branch exists, it returns the ELSE expression. Otherwise, NULL is returned.

In a searched CASE expression, the query evaluator searches from left to right until it finds a WHEN expression that is evaluated to TRUE, and then returns its corresponding THEN expression. If no condition is found to be TRUE, and an ELSE branch exists, it returns the ELSE expression. Otherwise, it returns NULL.

The following example illustrates the form of a case expression.

Example

+ +

CASE (2 < 3) WHEN true THEN "yes" ELSE "no" END
+

Quantified expressions

+ +

QuantifiedExpression ::= ( (<ANY>|<SOME>) | <EVERY> ) Variable <IN> Expression ( "," Variable "in" Expression )*
+                         <SATISFIES> Expression (<END>)?
+

Quantified expressions are used for expressing existential or universal predicates involving the elements of a collection.

The following pair of examples illustrate the use of a quantified expression to test that every (or some) element in the set [1, 2, 3] of integers is less than three. The first example yields FALSE and second example yields TRUE.

It is useful to note that if the set were instead the empty set, the first expression would yield TRUE (“every” value in an empty set satisfies the condition) while the second expression would yield FALSE (since there isn’t “some” value, as there are no values in the set, that satisfies the condition).

A quantified expression will return a NULL (or MISSING) if the first expression in it evaluates to NULL (or MISSING). A type error will be raised if the first expression in a quantified expression does not return a collection.

Examples

+ +

EVERY x IN [ 1, 2, 3 ] SATISFIES x < 3
+SOME x IN [ 1, 2, 3 ] SATISFIES x < 3
+

Path expressions

+ +

PathExpression  ::= PrimaryExpression ( Field | Index )*
+Field           ::= "." Identifier
+Index           ::= "[" ( Expression | "?" ) "]"
+

Components of complex types in the data model are accessed via path expressions. Path access can be applied to the result of a SQL++ expression that yields an instance of a complex type, e.g., a object or array instance. For objects, path access is based on field names. For arrays, path access is based on (zero-based) array-style indexing. SQL++ also supports an “I’m feeling lucky” style index accessor, [?], for selecting an arbitrary element from an array. Attempts to access non-existent fields or out-of-bound array elements produce the special value MISSING. Type errors will be raised for inappropriate use of a path expression, such as applying a field accessor to a numeric value.

The following examples illustrate field access for a object, index-based element access for an array, and also a composition thereof.

Examples

+ +

({"name": "MyABCs", "array": [ "a", "b", "c"]}).array
+
+(["a", "b", "c"])[2]
+
+({"name": "MyABCs", "array": [ "a", "b", "c"]}).array[2]
+

Primary Expressions

+ +

PrimaryExpr ::= Literal
+              | VariableReference
+              | ParenthesizedExpression
+              | FunctionCallExpression
+              | Constructor
+

The most basic building block for any SQL++ expression is PrimaryExpression. This can be a simple literal (constant) value, a reference to a query variable that is in scope, a parenthesized expression, a function call, or a newly constructed instance of the data model (such as a newly constructed object, array, or multiset of data model instances).

Literals

+ +

Literal        ::= StringLiteral
+                   | IntegerLiteral
+                   | FloatLiteral
+                   | DoubleLiteral
+                   | <NULL>
+                   | <MISSING>
+                   | <TRUE>
+                   | <FALSE>
+StringLiteral  ::= "\"" (
+                             <EscapeQuot>
+                           | <EscapeBslash>
+                           | <EscapeSlash>
+                           | <EscapeBspace>
+                           | <EscapeFormf>
+                           | <EscapeNl>
+                           | <EscapeCr>
+                           | <EscapeTab>
+                           | ~["\"","\\"])*
+                    "\""
+                    | "\'"(
+                             <EscapeApos>
+                           | <EscapeBslash>
+                           | <EscapeSlash>
+                           | <EscapeBspace>
+                           | <EscapeFormf>
+                           | <EscapeNl>
+                           | <EscapeCr>
+                           | <EscapeTab>
+                           | ~["\'","\\"])*
+                      "\'"
+<ESCAPE_Apos>  ::= "\\\'"
+<ESCAPE_Quot>  ::= "\\\""
+<EscapeBslash> ::= "\\\\"
+<EscapeSlash>  ::= "\\/"
+<EscapeBspace> ::= "\\b"
+<EscapeFormf>  ::= "\\f"
+<EscapeNl>     ::= "\\n"
+<EscapeCr>     ::= "\\r"
+<EscapeTab>    ::= "\\t"
+
+IntegerLiteral ::= <DIGITS>
+<DIGITS>       ::= ["0" - "9"]+
+FloatLiteral   ::= <DIGITS> ( "f" | "F" )
+                 | <DIGITS> ( "." <DIGITS> ( "f" | "F" ) )?
+                 | "." <DIGITS> ( "f" | "F" )
+DoubleLiteral  ::= <DIGITS> "." <DIGITS>
+                   | "." <DIGITS>
+

Literals (constants) in SQL++ can be strings, integers, floating point values, double values, boolean constants, or special constant values like NULL and MISSING. The NULL value is like a NULL in SQL; it is used to represent an unknown field value. The specialy value MISSING is only meaningful in the context of SQL++ field accesses; it occurs when the accessed field simply does not exist at all in a object being accessed.

The following are some simple examples of SQL++ literals.

Examples

+ +

'a string'
+"test string"
+42
+

Different from standard SQL, double quotes play the same role as single quotes and may be used for string literals in SQL++.

Variable References

+ +

VariableReference     ::= <IDENTIFIER>|<DelimitedIdentifier>
+<IDENTIFIER>          ::= <LETTER> (<LETTER> | <DIGIT> | "_" | "$")*
+<LETTER>              ::= ["A" - "Z", "a" - "z"]
+DelimitedIdentifier   ::= "`" (<EscapeQuot>
+                                | <EscapeBslash>
+                                | <EscapeSlash>
+                                | <EscapeBspace>
+                                | <EscapeFormf>
+                                | <EscapeNl>
+                                | <EscapeCr>
+                                | <EscapeTab>
+                                | ~["`","\\"])*
+                          "`"
+

A variable in SQL++ can be bound to any legal data model value. A variable reference refers to the value to which an in-scope variable is bound. (E.g., a variable binding may originate from one of the FROM, WITH or LET clauses of a SELECT statement or from an input parameter in the context of a function body.) Backticks, e.g., `id`, are used for delimited identifiers. Delimiting is needed when a variable’s desired name clashes with a SQL++ keyword or includes characters not allowed in regular identifiers.

Examples

+ +

tweet
+id
+`SELECT`
+`my-function`
+

Parenthesized expressions

+ +

ParenthesizedExpression ::= "(" Expression ")" | Subquery
+

An expression can be parenthesized to control the precedence order or otherwise clarify a query. In SQL++, for composability, a subquery is also an parenthesized expression.

The following expression evaluates to the value 2.

Example

+ +

( 1 + 1 )
+

Function call expressions

+ +

FunctionCallExpression ::= FunctionName "(" ( Expression ( "," Expression )* )? ")"
+

Functions are included in SQL++, like most languages, as a way to package useful functionality or to componentize complicated or reusable SQL++ computations. A function call is a legal SQL++ query expression that represents the value resulting from the evaluation of its body expression with the given parameter bindings; the parameter value bindings can themselves be any SQL++ expressions.

The following example is a (built-in) function call expression whose value is 8.

Example

+ +

length('a string')
+

Constructors

+ +

Constructor              ::= ArrayConstructor | MultisetConstructor | ObjectConstructor
+ArrayConstructor         ::= "[" ( Expression ( "," Expression )* )? "]"
+MultisetConstructor      ::= "{{" ( Expression ( "," Expression )* )? "}}"
+ObjectConstructor        ::= "{" ( FieldBinding ( "," FieldBinding )* )? "}"
+FieldBinding             ::= Expression ":" Expression
+

A major feature of SQL++ is its ability to construct new data model instances. This is accomplished using its constructors for each of the model’s complex object structures, namely arrays, multisets, and objects. Arrays are like JSON arrays, while multisets have bag semantics. Objects are built from fields that are field-name/field-value pairs, again like JSON.

The following examples illustrate how to construct a new array with 4 items, a new object with 2 fields, and a new multiset with 5 items, respectively. Array elements or multiset elements can be homogeneous (as in the first example), which is the common case, or they may be heterogeneous (as in the third example). The data values and field name values used to construct arrays, multisets, and objects in constructors are all simply SQL++ expressions. Thus, the collection elements, field names, and field values used in constructors can be simple literals or they can come from query variable references or even arbitrarily complex SQL++ expressions (subqueries). Type errors will be raised if the field names in a record must be strings, and duplicate field errors will be raised if they are not distinct.

Examples

+ +

[ 'a', 'b', 'c', 'c' ]
+
+{
+  'project name': 'Hyracks',
+  'project members': [ 'vinayakb', 'dtabass', 'chenli', 'tsotras', 'tillw' ]
+}
+
+{{ 42, "forty-two!", { "rank": "Captain", "name": "America" }, 3.14159, 42 }}
+

+ +

3. Queries

A SQL++ query can be any legal SQL++ expression or SELECT statement. A SQL++ query always ends with a semicolon.

+ +

Query ::= (Expression | SelectStatement) ";"
+

SELECT statements

The following shows the (rich) grammar for the SELECT statement in SQL++.

+ +

SelectStatement    ::= ( WithClause )?
+                       SelectSetOperation (OrderbyClause )? ( LimitClause )?
+SelectSetOperation ::= SelectBlock (<UNION> <ALL> ( SelectBlock | Subquery ) )*
+Subquery           ::= "(" SelectStatement ")"
+
+SelectBlock        ::= SelectClause
+                       ( FromClause ( LetClause )?)?
+                       ( WhereClause )?
+                       ( GroupbyClause ( LetClause )? ( HavingClause )? )?
+                       |
+                       FromClause ( LetClause )?
+                       ( WhereClause )?
+                       ( GroupbyClause ( LetClause )? ( HavingClause )? )?
+                       SelectClause
+
+SelectClause       ::= <SELECT> ( <ALL> | <DISTINCT> )? ( SelectRegular | SelectValue )
+SelectRegular      ::= Projection ( "," Projection )*
+SelectValue      ::= ( <VALUE> | <ELEMENT> | <RAW> ) Expression
+Projection         ::= ( Expression ( <AS> )? Identifier | "*" )
+
+FromClause         ::= <FROM> FromTerm ( "," FromTerm )*
+FromTerm           ::= Expression (( <AS> )? Variable)?
+                       ( ( JoinType )? ( JoinClause | UnnestClause ) )*
+
+JoinClause         ::= <JOIN> Expression (( <AS> )? Variable)? <ON> Expression
+UnnestClause       ::= ( <UNNEST> | <CORRELATE> | <FLATTEN> ) Expression
+                       ( <AS> )? Variable ( <AT> Variable )?
+JoinType           ::= ( <INNER> | <LEFT> ( <OUTER> )? )
+
+WithClause         ::= <WITH> WithElement ( "," WithElement )*
+LetClause          ::= (<LET> | <LETTING>) LetElement ( "," LetElement )*
+LetElement         ::= Variable "=" Expression
+WithElement        ::= Variable <AS> Expression
+
+WhereClause        ::= <WHERE> Expression
+
+GroupbyClause      ::= <GROUP> <BY> ( Expression ( (<AS>)? Variable )? ( "," Expression ( (<AS>)? Variable )? )*
+                       ( <GROUP> <AS> Variable
+                         ("(" Variable <AS> VariableReference ("," Variable <AS> VariableReference )* ")")?
+                       )?
+HavingClause       ::= <HAVING> Expression
+
+OrderbyClause      ::= <ORDER> <BY> Expression ( <ASC> | <DESC> )? ( "," Expression ( <ASC> | <DESC> )? )*
+LimitClause        ::= <LIMIT> Expression ( <OFFSET> Expression )?
+

In this section, we will make use of two stored collections of objects (datasets), GleambookUsers and GleambookMessages, in a series of running examples to explain SELECT queries. The contents of the example collections are as follows:

GleambookUsers collection (or, dataset):

+ +

{"id":1,"alias":"Margarita","name":"MargaritaStoddard","nickname":"Mags","userSince":"2012-08-20T10:10:00","friendIds":[2,3,6,10],"employment":[{"organizationName":"Codetechno","start-date":"2006-08-06"},{"organizationName":"geomedia","start-date":"2010-06-17","end-date":"2010-01-26"}],"gender":"F"}
+{"id":2,"alias":"Isbel","name":"IsbelDull","nickname":"Izzy","userSince":"2011-01-22T10:10:00","friendIds":[1,4],"employment":[{"organizationName":"Hexviafind","startDate":"2010-04-27"}]}
+{"id":3,"alias":"Emory","name":"EmoryUnk","userSince":"2012-07-10T10:10:00","friendIds":[1,5,8,9],"employment":[{"organizationName":"geomedia","startDate":"2010-06-17","endDate":"2010-01-26"}]}
+

GleambookMessages collection (or, dataset):

+ +

{"messageId":2,"authorId":1,"inResponseTo":4,"senderLocation":[41.66,80.87],"message":" dislike iphone its touch-screen is horrible"}
+{"messageId":3,"authorId":2,"inResponseTo":4,"senderLocation":[48.09,81.01],"message":" like samsung the plan is amazing"}
+{"messageId":4,"authorId":1,"inResponseTo":2,"senderLocation":[37.73,97.04],"message":" can't stand at&t the network is horrible:("}
+{"messageId":6,"authorId":2,"inResponseTo":1,"senderLocation":[31.5,75.56],"message":" like t-mobile its platform is mind-blowing"}
+{"messageId":8,"authorId":1,"inResponseTo":11,"senderLocation":[40.33,80.87],"message":" like verizon the 3G is awesome:)"}
+{"messageId":10,"authorId":1,"inResponseTo":12,"senderLocation":[42.5,70.01],"message":" can't stand motorola the touch-screen is terrible"}
+{"messageId":11,"authorId":1,"inResponseTo":1,"senderLocation":[38.97,77.49],"message":" can't stand at&t its plan is terrible"}
+

SELECT Clause

The SQL++ SELECT clause always returns a collection value as its result (even if the result is empty or a singleton).

SELECT VALUE Clause

The SELECT VALUE clause in SQL++ returns a collection that contains the results of evaluating the VALUE expression, with one evaluation being performed per “binding tuple” (i.e., per FROM clause item) satisfying the statement’s selection criteria. For historical reasons SQL++ also allows the keywords ELEMENT or RAW to be used in place of VALUE (not recommended).

The following example shows a standard-alone SELECT VALUE, which wraps a value into an array.

Example

+ +

SELECT VALUE 1;
+

This query return:

+ +

[
+  1
+]
+

The following example shows a query that selects one user from the GleambookUsers collection.

Example

+ +

SELECT VALUE user
+FROM GleambookUsers user
+WHERE user.id = 1;
+

This query returns:

+ +

[{
+    "userSince": "2012-08-20T10:10:00.000Z",
+    "friendIds": [
+        2,
+        3,
+        6,
+        10
+    ],
+    "gender": "F",
+    "name": "MargaritaStoddard",
+    "nickname": "Mags",
+    "alias": "Margarita",
+    "id": 1,
+    "employment": [
+        {
+            "organizationName": "Codetechno",
+            "start-date": "2006-08-06"
+        },
+        {
+            "end-date": "2010-01-26",
+            "organizationName": "geomedia",
+            "start-date": "2010-06-17"
+        }
+    ]
+} ]
+

SQL-style SELECT

In SQL++, the traditional SQL-style SELECT syntax is also supported. This syntax can also be reformulated in a SELECT VALUE based manner in SQL++. (E.g., SELECT expA AS fldA, expB AS fldB is syntactic sugar for SELECT VALUE { 'fldA': expA, 'fldB': expB }.) Unlike in SQL, the result of an SQL++ query does not preserve the order of expressions in the SELECT clause.

Example

+ +

SELECT user.alias user_alias, user.name user_name
+FROM GleambookUsers user
+WHERE user.id = 1;
+

Returns:

+ +

[ {
+    "user_name": "MargaritaStoddard",
+    "user_alias": "Margarita"
+} ]
+

SELECT *

In SQL++, SELECT * returns a object with a nested field for each input tuple. Each field has as its field name the name of a binding variable generated by either the FROM clause or GROUP BY clause in the current enclosing SELECT statement, and its field value is the value of that binding variable.

Example

+ +

SELECT *
+FROM GleambookUsers user;
+

Since user is the only binding variable generated in the FROM clause, this query returns:

+ +

[ {
+    "user": {
+        "userSince": "2012-08-20T10:10:00.000Z",
+        "friendIds": [
+            2,
+            3,
+            6,
+            10
+        ],
+        "gender": "F",
+        "name": "MargaritaStoddard",
+        "nickname": "Mags",
+        "alias": "Margarita",
+        "id": 1,
+        "employment": [
+            {
+                "organizationName": "Codetechno",
+                "start-date": "2006-08-06"
+            },
+            {
+                "end-date": "2010-01-26",
+                "organizationName": "geomedia",
+                "start-date": "2010-06-17"
+            }
+        ]
+    }
+}, {
+    "user": {
+        "userSince": "2011-01-22T10:10:00.000Z",
+        "friendIds": [
+            1,
+            4
+        ],
+        "name": "IsbelDull",
+        "nickname": "Izzy",
+        "alias": "Isbel",
+        "id": 2,
+        "employment": [
+            {
+                "organizationName": "Hexviafind",
+                "startDate": "2010-04-27"
+            }
+        ]
+    }
+}, {
+    "user": {
+        "userSince": "2012-07-10T10:10:00.000Z",
+        "friendIds": [
+            1,
+            5,
+            8,
+            9
+        ],
+        "name": "EmoryUnk",
+        "alias": "Emory",
+        "id": 3,
+        "employment": [
+            {
+                "organizationName": "geomedia",
+                "endDate": "2010-01-26",
+                "startDate": "2010-06-17"
+            }
+        ]
+    }
+} ]
+

Example

+ +

SELECT *
+FROM GleambookUsers u, GleambookMessages m
+WHERE m.authorId = u.id and u.id = 2;
+

This query does an inner join that we will discuss in multiple from terms. Since both u and m are binding variable generated in the FROM clause, this query returns:

+ +

[ {
+    "u": {
+        "userSince": "2011-01-22T10:10:00",
+        "friendIds": [
+            1,
+            4
+        ],
+        "name": "IsbelDull",
+        "nickname": "Izzy",
+        "alias": "Isbel",
+        "id": 2,
+        "employment": [
+            {
+                "organizationName": "Hexviafind",
+                "startDate": "2010-04-27"
+            }
+        ]
+    },
+    "m": {
+        "senderLocation": [
+            31.5,
+            75.56
+        ],
+        "inResponseTo": 1,
+        "messageId": 6,
+        "authorId": 2,
+        "message": " like t-mobile its platform is mind-blowing"
+    }
+}, {
+    "u": {
+        "userSince": "2011-01-22T10:10:00",
+        "friendIds": [
+            1,
+            4
+        ],
+        "name": "IsbelDull",
+        "nickname": "Izzy",
+        "alias": "Isbel",
+        "id": 2,
+        "employment": [
+            {
+                "organizationName": "Hexviafind",
+                "startDate": "2010-04-27"
+            }
+        ]
+    },
+    "m": {
+        "senderLocation": [
+            48.09,
+            81.01
+        ],
+        "inResponseTo": 4,
+        "messageId": 3,
+        "authorId": 2,
+        "message": " like samsung the plan is amazing"
+    }
+} ]
+

SELECT DISTINCT

SQL++’s DISTINCT keyword is used to eliminate duplicate items in results. The following example shows how it works.

Example

+ +

SELECT DISTINCT * FROM [1, 2, 2, 3] AS foo;
+

This query returns:

+ +

[ {
+    "foo": 1
+}, {
+    "foo": 2
+}, {
+    "foo": 3
+} ]
+

Example

+ +

SELECT DISTINCT VALUE foo FROM [1, 2, 2, 3] AS foo;
+

This version of the query returns:

+ +

[ 1
+, 2
+, 3
+ ]
+

Unnamed projections

Similar to standard SQL, SQL++ supports unnamed projections (a.k.a, unnamed SELECT clause items), for which names are generated. Name generation has three cases:

+ +

If a projection expression is a variable reference expression, its generated name is the name of the variable.
If a projection expression is a field access expression, its generated name is the last identifier in the expression.
For all other cases, the query processor will generate a unique name.

Example

+ +

SELECT substr(user.name, 10), user.alias
+FROM GleambookUsers user
+WHERE user.id = 1;
+

This query outputs:

+ +

[ {
+    "alias": "Margarita",
+    "$1": "Stoddard"
+} ]
+

In the result, $1 is the generated name for substr(user.name, 1), while alias is the generated name for user.alias.

Abbreviated Field Access Expressions

As in standard SQL, SQL++ field access expressions can be abbreviated (not recommended) when there is no ambiguity. In the next example, the variable user is the only possible variable reference for fields id, name and alias and thus could be omitted in the query.

Example

+ +

SELECT substr(name, 10) AS lname, alias
+FROM GleambookUsers user
+WHERE id = 1;
+

Outputs:

+ +

[ {
+    "lname": "Stoddard",
+    "alias": "Margarita"
+} ]
+

UNNEST Clause

For each of its input tuples, the UNNEST clause flattens a collection-valued expression into individual items, producing multiple tuples, each of which is one of the expression’s original input tuples augmented with a flattened item from its collection.

Inner UNNEST

The following example is a query that retrieves the names of the organizations that a selected user has worked for. It uses the UNNEST clause to unnest the nested collection employment in the user’s object.

Example

+ +

SELECT u.id AS userId, e.organizationName AS orgName
+FROM GleambookUsers u
+UNNEST u.employment e
+WHERE u.id = 1;
+

This query returns:

+ +

[ {
+    "orgName": "Codetechno",
+    "userId": 1
+}, {
+    "orgName": "geomedia",
+    "userId": 1
+} ]
+

Note that UNNEST has SQL’s inner join semantics — that is, if a user has no employment history, no tuple corresponding to that user will be emitted in the result.

Left outer UNNEST

As an alternative, the LEFT OUTER UNNEST clause offers SQL’s left outer join semantics. For example, no collection-valued field named hobbies exists in the object for the user whose id is 1, but the following query’s result still includes user 1.

Example

+ +

SELECT u.id AS userId, h.hobbyName AS hobby
+FROM GleambookUsers u
+LEFT OUTER UNNEST u.hobbies h
+WHERE u.id = 1;
+

Returns:

+ +

[ {
+    "userId": 1
+} ]
+

Note that if u.hobbies is an empty collection or leads to a MISSING (as above) or NULL value for a given input tuple, there is no corresponding binding value for variable h for an input tuple. A MISSING value will be generated for h so that the input tuple can still be propagated.

Expressing joins using UNNEST

The SQL++ UNNEST clause is similar to SQL’s JOIN clause except that it allows its right argument to be correlated to its left argument, as in the examples above — i.e., think “correlated cross-product”. The next example shows this via a query that joins two data sets, GleambookUsers and GleambookMessages, returning user/message pairs. The results contain one object per pair, with result objects containing the user’s name and an entire message. The query can be thought of as saying “for each Gleambook user, unnest the GleambookMessages collection and filter the output with the condition message.authorId = user.id”.

Example

+ +

SELECT u.name AS uname, m.message AS message
+FROM GleambookUsers u
+UNNEST GleambookMessages m
+WHERE m.authorId = u.id;
+

This returns:

+ +

[ {
+    "uname": "MargaritaStoddard",
+    "message": " can't stand at&t its plan is terrible"
+}, {
+    "uname": "MargaritaStoddard",
+    "message": " dislike iphone its touch-screen is horrible"
+}, {
+    "uname": "MargaritaStoddard",
+    "message": " can't stand at&t the network is horrible:("
+}, {
+    "uname": "MargaritaStoddard",
+    "message": " like verizon the 3G is awesome:)"
+}, {
+    "uname": "MargaritaStoddard",
+    "message": " can't stand motorola the touch-screen is terrible"
+}, {
+    "uname": "IsbelDull",
+    "message": " like t-mobile its platform is mind-blowing"
+}, {
+    "uname": "IsbelDull",
+    "message": " like samsung the plan is amazing"
+} ]
+

Similarly, the above query can also be expressed as the UNNESTing of a correlated SQL++ subquery:

Example

+ +

SELECT u.name AS uname, m.message AS message
+FROM GleambookUsers u
+UNNEST (
+    SELECT VALUE msg
+    FROM GleambookMessages msg
+    WHERE msg.authorId = u.id
+) AS m;
+

FROM clauses

A FROM clause is used for enumerating (i.e., conceptually iterating over) the contents of collections, as in SQL.

Binding expressions

In SQL++, in addition to stored collections, a FROM clause can iterate over any intermediate collection returned by a valid SQL++ expression. In the tuple stream generated by a FROM clause, the ordering of the input tuples are not guaranteed to be preserved.

Example

+ +

SELECT VALUE foo
+FROM [1, 2, 2, 3] AS foo
+WHERE foo > 2;
+

Returns:

+ +

[
+  3
+]
+

Multiple FROM terms

SQL++ permits correlations among FROM terms. Specifically, a FROM binding expression can refer to variables defined to its left in the given FROM clause. Thus, the first unnesting example above could also be expressed as follows:

Example

+ +

SELECT u.id AS userId, e.organizationName AS orgName
+FROM GleambookUsers u, u.employment e
+WHERE u.id = 1;
+

Expressing joins using FROM terms

Similarly, the join intentions of the other UNNEST-based join examples above could be expressed as:

Example

+ +

SELECT u.name AS uname, m.message AS message
+FROM GleambookUsers u, GleambookMessages m
+WHERE m.authorId = u.id;
+

Example

+ +

SELECT u.name AS uname, m.message AS message
+FROM GleambookUsers u,
+  (
+    SELECT VALUE msg
+    FROM GleambookMessages msg
+    WHERE msg.authorId = u.id
+  ) AS m;
+

Note that the first alternative is one of the SQL-92 approaches to expressing a join.

Implicit binding variables

Similar to standard SQL, SQL++ supports implicit FROM binding variables (i.e., aliases), for which a binding variable is generated. SQL++ variable generation falls into three cases:

+ +

If the binding expression is a variable reference expression, the generated variable’s name will be the name of the referenced variable itself.
If the binding expression is a field access expression (or a fully qualified name for a dataset), the generated variable’s name will be the last identifier (or the dataset name) in the expression.
For all other cases, a compilation error will be raised.

The next two examples show queries that do not provide binding variables in their FROM clauses.

Example

+ +

SELECT GleambookUsers.name, GleambookMessages.message
+FROM GleambookUsers, GleambookMessages
+WHERE GleambookMessages.authorId = GleambookUsers.id;
+

Returns:

+ +

[ {
+    "name": "MargaritaStoddard",
+    "message": " like verizon the 3G is awesome:)"
+}, {
+    "name": "MargaritaStoddard",
+    "message": " can't stand motorola the touch-screen is terrible"
+}, {
+    "name": "MargaritaStoddard",
+    "message": " can't stand at&t its plan is terrible"
+}, {
+    "name": "MargaritaStoddard",
+    "message": " dislike iphone its touch-screen is horrible"
+}, {
+    "name": "MargaritaStoddard",
+    "message": " can't stand at&t the network is horrible:("
+}, {
+    "name": "IsbelDull",
+    "message": " like samsung the plan is amazing"
+}, {
+    "name": "IsbelDull",
+    "message": " like t-mobile its platform is mind-blowing"
+} ]
+

Example

+ +

SELECT GleambookUsers.name, GleambookMessages.message
+FROM GleambookUsers,
+  (
+    SELECT VALUE GleambookMessages
+    FROM GleambookMessages
+    WHERE GleambookMessages.authorId = GleambookUsers.id
+  );
+

Returns:

+ +

Error: "Syntax error: Need an alias for the enclosed expression:\n(select element GleambookMessages\n    from GleambookMessages as GleambookMessages\n    where (GleambookMessages.authorId = GleambookUsers.id)\n )",
+    "query_from_user": "use TinySocial;\n\nSELECT GleambookUsers.name, GleambookMessages.message\n    FROM GleambookUsers,\n      (\n        SELECT VALUE GleambookMessages\n        FROM GleambookMessages\n        WHERE GleambookMessages.authorId = GleambookUsers.id\n      );"
+

JOIN clauses

The join clause in SQL++ supports both inner joins and left outer joins from standard SQL.

Inner joins

Using a JOIN clause, the inner join intent from the preceeding examples can also be expressed as follows:

Example

+ +

SELECT u.name AS uname, m.message AS message
+FROM GleambookUsers u JOIN GleambookMessages m ON m.authorId = u.id;
+

Left outer joins

SQL++ supports SQL’s notion of left outer join. The following query is an example:

+ +

SELECT u.name AS uname, m.message AS message
+FROM GleambookUsers u LEFT OUTER JOIN GleambookMessages m ON m.authorId = u.id;
+

Returns:

+ +

[ {
+    "uname": "MargaritaStoddard",
+    "message": " like verizon the 3G is awesome:)"
+}, {
+    "uname": "MargaritaStoddard",
+    "message": " can't stand motorola the touch-screen is terrible"
+}, {
+    "uname": "MargaritaStoddard",
+    "message": " can't stand at&t its plan is terrible"
+}, {
+    "uname": "MargaritaStoddard",
+    "message": " dislike iphone its touch-screen is horrible"
+}, {
+    "uname": "MargaritaStoddard",
+    "message": " can't stand at&t the network is horrible:("
+}, {
+    "uname": "IsbelDull",
+    "message": " like samsung the plan is amazing"
+}, {
+    "uname": "IsbelDull",
+    "message": " like t-mobile its platform is mind-blowing"
+}, {
+    "uname": "EmoryUnk"
+} ]
+

For non-matching left-side tuples, SQL++ produces MISSING values for the right-side binding variables; that is why the last object in the above result doesn’t have a message field. Note that this is slightly different from standard SQL, which instead would fill in NULL values for the right-side fields. The reason for this difference is that, for non-matches in its join results, SQL++ views fields from the right-side as being “not there” (a.k.a. MISSING) instead of as being “there but unknown” (i.e., NULL).

The left-outer join query can also be expressed using LEFT OUTER UNNEST:

+ +

SELECT u.name AS uname, m.message AS message
+FROM GleambookUsers u
+LEFT OUTER UNNEST (
+    SELECT VALUE message
+    FROM GleambookMessages message
+    WHERE message.authorId = u.id
+  ) m;
+

In general, in SQL++, SQL-style join queries can also be expressed by UNNEST clauses and left outer join queries can be expressed by LEFT OUTER UNNESTs.

GROUP BY clauses

The SQL++ GROUP BY clause generalizes standard SQL’s grouping and aggregation semantics, but it also retains backward compatibility with the standard (relational) SQL GROUP BY and aggregation features.

Group variables

In a GROUP BY clause, in addition to the binding variable(s) defined for the grouping key(s), SQL++ allows a user to define a group variable by using the clause’s GROUP AS extension to denote the resulting group. After grouping, then, the query’s in-scope variables include the grouping key’s binding variables as well as this group variable which will be bound to one collection value for each group. This per-group collection (i.e., multiset) value will be a set of nested objects in which each field of the object is the result of a renamed variable defined in parentheses following the group variable’s name. The GROUP AS syntax is as follows:

+ +

<GROUP> <AS> Variable ("(" Variable <AS> VariableReference ("," Variable <AS> VariableReference )* ")")?
+

Example

+ +

SELECT *
+FROM GleambookMessages message
+GROUP BY message.authorId AS uid GROUP AS msgs(message AS msg);
+

This first example query returns:

+ +

[ {
+    "msgs": [
+        {
+            "msg": {
+                "senderLocation": [
+                    38.97,
+                    77.49
+                ],
+                "inResponseTo": 1,
+                "messageId": 11,
+                "authorId": 1,
+                "message": " can't stand at&t its plan is terrible"
+            }
+        },
+        {
+            "msg": {
+                "senderLocation": [
+                    41.66,
+                    80.87
+                ],
+                "inResponseTo": 4,
+                "messageId": 2,
+                "authorId": 1,
+                "message": " dislike iphone its touch-screen is horrible"
+            }
+        },
+        {
+            "msg": {
+                "senderLocation": [
+                    37.73,
+                    97.04
+                ],
+                "inResponseTo": 2,
+                "messageId": 4,
+                "authorId": 1,
+                "message": " can't stand at&t the network is horrible:("
+            }
+        },
+        {
+            "msg": {
+                "senderLocation": [
+                    40.33,
+                    80.87
+                ],
+                "inResponseTo": 11,
+                "messageId": 8,
+                "authorId": 1,
+                "message": " like verizon the 3G is awesome:)"
+            }
+        },
+        {
+            "msg": {
+                "senderLocation": [
+                    42.5,
+                    70.01
+                ],
+                "inResponseTo": 12,
+                "messageId": 10,
+                "authorId": 1,
+                "message": " can't stand motorola the touch-screen is terrible"
+            }
+        }
+    ],
+    "uid": 1
+}, {
+    "msgs": [
+        {
+            "msg": {
+                "senderLocation": [
+                    31.5,
+                    75.56
+                ],
+                "inResponseTo": 1,
+                "messageId": 6,
+                "authorId": 2,
+                "message": " like t-mobile its platform is mind-blowing"
+            }
+        },
+        {
+            "msg": {
+                "senderLocation": [
+                    48.09,
+                    81.01
+                ],
+                "inResponseTo": 4,
+                "messageId": 3,
+                "authorId": 2,
+                "message": " like samsung the plan is amazing"
+            }
+        }
+    ],
+    "uid": 2
+} ]
+

As we can see from the above query result, each group in the example query’s output has an associated group variable value called msgs that appears in the SELECT *’s result. This variable contains a collection of objects associated with the group; each of the group’s message values appears in the msg field of the objects in the msgs collection.

The group variable in SQL++ makes more complex, composable, nested subqueries over a group possible, which is important given the more complex data model of SQL++ (relative to SQL). As a simple example of this, as we really just want the messages associated with each user, we might wish to avoid the “extra wrapping” of each message as the msg field of a object. (That wrapping is useful in more complex cases, but is essentially just in the way here.) We can use a subquery in the SELECT clase to tunnel through the extra nesting and produce the desired result.

Example

+ +

SELECT uid, (SELECT VALUE g.msg FROM g) AS msgs
+FROM GleambookMessages gbm
+GROUP BY gbm.authorId AS uid
+GROUP AS g(gbm as msg);
+

This variant of the example query returns:

+ +

   [ {
+       "msgs": [
+           {
+               "senderLocation": [
+                   38.97,
+                   77.49
+               ],
+               "inResponseTo": 1,
+               "messageId": 11,
+               "authorId": 1,
+               "message": " can't stand at&t its plan is terrible"
+           },
+           {
+               "senderLocation": [
+                   41.66,
+                   80.87
+               ],
+               "inResponseTo": 4,
+               "messageId": 2,
+               "authorId": 1,
+               "message": " dislike iphone its touch-screen is horrible"
+           },
+           {
+               "senderLocation": [
+                   37.73,
+                   97.04
+               ],
+               "inResponseTo": 2,
+               "messageId": 4,
+               "authorId": 1,
+               "message": " can't stand at&t the network is horrible:("
+           },
+           {
+               "senderLocation": [
+                   40.33,
+                   80.87
+               ],
+               "inResponseTo": 11,
+               "messageId": 8,
+               "authorId": 1,
+               "message": " like verizon the 3