drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bridg...@apache.org
Subject [1/2] drill git commit: Drill 1.11 Encryption and Security Content from Catherine Skrbina/Drill 1.11 Identifier Quotes Content from Bridget Bevens
Date Mon, 07 Aug 2017 19:05:07 GMT
Repository: drill
Updated Branches:
  refs/heads/gh-pages 7cab70c9f -> 25aa6a18e


http://git-wip-us.apache.org/repos/asf/drill/blob/25aa6a18/_docs/developer-information/design-docs/020-rpc-overview.md
----------------------------------------------------------------------
diff --git a/_docs/developer-information/design-docs/020-rpc-overview.md b/_docs/developer-information/design-docs/020-rpc-overview.md
index 4d4ca09..0cb641b 100644
--- a/_docs/developer-information/design-docs/020-rpc-overview.md
+++ b/_docs/developer-information/design-docs/020-rpc-overview.md
@@ -1,6 +1,6 @@
 ---
 title: "RPC Overview"
-date:  
+date: 2017-08-07 19:02:50 UTC
 parent: "Design Docs"
 ---
 Drill leverages the Netty 4 project as an RPC underlayment. From there, we
@@ -18,3 +18,26 @@ RpcHeader has the following fields:
 
 Drillbits communicate through the BitCom intermediary. BitCom manages...
 
+##Drill Channel Pipeline with Handlers
+The Drill RPC layer is built on [Netty](http://netty.io/index.html "Netty project"), an asynchronous
network application framework that makes it easy to develop network-related components (such
as, client and server) in an application. Within Netty, each connection is represented as
a *Channel* that consists of its own *Pipeline*. The Pipeline is created when a Channel is
created.  A Channel Pipeline consists of one or more inbound and/or outbound ChannelHandlers,
which act on I/O events sent and/or received by an application. (For reference, see [Interface
Channel](https://netty.io/4.0/api/io/netty/channel/Channel.html "Interface Channel"), [Interface
ChannelPipeline](https://netty.io/4.0/api/io/netty/channel/ChannelPipeline.html "Interface
ChannelPipeline"), and [Interface ChannelHandler](https://netty.io/4.0/api/io/netty/channel/ChannelHandler.html
"Interface ChannelHandler").)
+
+In the Drill ecosystem, UserClient (on the client side) and UserServer (on the server side)
represent the wrapper for each connection between client and server, respectively.  They define
the Pipeline and various handlers added for the communication path. 
+
+For encryption support, both UserClient and UserServer require modification since new handlers
will be added for encryption and decryption if privacy is negotiated as part of the handshake.

+
+###Encryption, Decryption, and ChunkCreation Handlers
+In addition to an Encryption/Decryption handler, a ChunkCreation handler on the sender side
and LengthFieldBasedFrameDecoder on the receiver side should be added. 
+
+The ChunkCreation handler helps divide the Outgoing RPC message into smaller chunk units
for encryption, whereas the LengthFieldBasedFrameDecoder helps to accumulate all  bytes of
an encrypted payload on the receiver side before calling the decrypt module on it. 
+
+The following diagram shows the existing handlers as well as the new handlers added in the
Drill Channel Pipeline for client-to-drillbit encryption support in Drill 1.11. 
+
+![drillpipeline]({{site.baseurl}}/docs/img/drill-channel-pipeline-with-handlers.png)  
+
+######Drill Channel Pipeline with Handlers
+
+
+
+
+
+

http://git-wip-us.apache.org/repos/asf/drill/blob/25aa6a18/_docs/img/client-encrypt-compatibility.png
----------------------------------------------------------------------
diff --git a/_docs/img/client-encrypt-compatibility.png b/_docs/img/client-encrypt-compatibility.png
new file mode 100644
index 0000000..4b43046
Binary files /dev/null and b/_docs/img/client-encrypt-compatibility.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/25aa6a18/_docs/img/drill-channel-pipeline-with-handlers.png
----------------------------------------------------------------------
diff --git a/_docs/img/drill-channel-pipeline-with-handlers.png b/_docs/img/drill-channel-pipeline-with-handlers.png
new file mode 100644
index 0000000..99aca9a
Binary files /dev/null and b/_docs/img/drill-channel-pipeline-with-handlers.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/25aa6a18/_docs/img/kerberos-clnt-svr.png
----------------------------------------------------------------------
diff --git a/_docs/img/kerberos-clnt-svr.png b/_docs/img/kerberos-clnt-svr.png
new file mode 100644
index 0000000..1012fda
Binary files /dev/null and b/_docs/img/kerberos-clnt-svr.png differ

http://git-wip-us.apache.org/repos/asf/drill/blob/25aa6a18/_docs/sql-reference/030-lexical-structure.md
----------------------------------------------------------------------
diff --git a/_docs/sql-reference/030-lexical-structure.md b/_docs/sql-reference/030-lexical-structure.md
index 06c6a80..7314258 100644
--- a/_docs/sql-reference/030-lexical-structure.md
+++ b/_docs/sql-reference/030-lexical-structure.md
@@ -1,6 +1,6 @@
 ---
 title: "Lexical Structure"
-date:  
+date: 2017-08-07 19:02:56 UTC
 parent: "SQL Reference"
 ---
 
@@ -74,13 +74,13 @@ Format dates using dashes (-) to separate year, month, and day. Format
time usin
 
 If you have dates and times in other formats, use a [data type conversion function](/data-type-conversion/#other-data-type-conversions)
in your queries.
 
-### Identifier
-An identifier is a letter followed by any sequence of letters, digits, or the underscore.
For example, names of tables, columns, and aliases are identifiers. Maximum length is 1024
characters. Enclose the following identifiers in back ticks:
+### Identifiers
+An identifier is a letter followed by any sequence of letters, digits, or the underscore.
For example, names of tables, columns, and aliases are identifiers. Maximum length is 1024
characters. Enclose the following identifiers with identifier quotes:
 
 * Keywords
-* Identifiers that SQL cannot parse. 
+* Identifiers that SQL cannot parse
 
-For example, enclose the SQL keywords date and time in back ticks when referring to column
names, but not when referring to data types:
+For example, enclose the SQL keywords date and time in identifier quotes when referring to
column names, but not when referring to data types:
 
     CREATE TABLE dfs.tmp.sampleparquet AS 
     (SELECT trans_id, 
@@ -88,14 +88,13 @@ For example, enclose the SQL keywords date and time in back ticks when
referring
     cast(`time` AS time) transtime, 
     cast(amount AS double) amountm,
     user_info, marketing_info, trans_info 
-    FROM dfs.`/Users/drilluser/sample.json`);
+    FROM dfs.`/Users/drilluser/sample.json`);    
 
-Table and column names are case-insensitive. Use back ticks to enclose names that contain
special characters. Special characters are those other than the 52 Latin alphabet characters.
For example, space and @ are special characters. 
+Table and column names are case-insensitive. Use identifier quotes to enclose names that
contain special characters. Special characters are those other than the 52 Latin alphabet
characters. For example, space and @ are special characters. 
 
-The following example shows the keyword Year enclosed in back ticks. Because the column alias
contains the special space character, also enclose the alias in back ticks, as shown in the
following example:
+The following example shows the keyword Year enclosed in identifier quotes. Because the column
alias contains the special space character, also enclose the alias in back ticks, as shown
in the following example:
 
     SELECT extract(year from transdate) AS `Year`, t.user_info.cust_id AS `Customer Number`
FROM dfs.tmp.`sampleparquet` t;
-
     +------------+-----------------+
     |    Year    | Customer Number |
     +------------+-----------------+
@@ -105,7 +104,55 @@ The following example shows the keyword Year enclosed in back ticks.
Because the
     | 2013       | 666             |
     | 2013       | 999             |
     +------------+-----------------+
-    5 rows selected (0.051 seconds)
+    5 rows selected (0.051 seconds)  
+
+
+### Identifier Quotes
+Prior to Drill 1.11, the SQL parser in Drill only supported back ticks as identifier quotes.
As of Drill 1.11, the SQL parser can also use double quotes and square brackets. The default
setting for identifier quotes is back ticks. You can configure the type of identifier quotes
used with the  `planner.parser.quoting_identifiers` configuration option, at the system or
session level, as shown:  
+
+       ALTER SYSTEM|SESSION SET planner.parser.quoting_identifiers = '"';  
+       ALTER SYSTEM|SESSION SET planner.parser.quoting_identifiers = '[';  
+       ALTER SYSTEM|SESSION SET planner.parser.quoting_identifiers = '`';  
+
+The following table lists the supported identifier quotes with their corresponding Unicode
character:   
+ 
+| Quoting   Identifier | Unicode   Character                                            
      |
+|----------------------|-----------------------------------------------------------------------|
+| Back ticks           | 'GRAVE   ACCENT' (U+0060)                                      
      |
+| Double quotes        | 'QUOTATION   MARK' (U+0022)                                    
      |
+| Square brackets      | 'LEFT   SQUARE BRACKET' (U+005B) and 'RIGHT SQUARE BRACKET' (U+005D;)
|  
+
+Alternatively, you can set the type of identifier using the `quoting_identifiers` property
in the jdbc connection URL, as shown:  
+ 
+       jdbc:drill:zk=local;quoting_identifiers=[  
+
+**Note:** The identifier quotes used in queries must match the `planner.parser.quoting_identifiers`
setting. If you use another type of identifier quotes, Drill returns an error.  
+
+The following queries show the use of each type of identifier quotes:  
+
+       0: jdbc:drill:zk=local> select `employee_id`, `full_name` from cp.`employee.json`
limit 1;
+       +--------------+---------------+
+       | employee_id  |   full_name   |
+       +--------------+---------------+
+       | 1            | Sheri Nowmer  |
+       +--------------+---------------+
+       1 row selected (0.148 seconds)  
+
+       0: jdbc:drill:zk=local> select "employee_id", "full_name" from cp."employee.json"
limit 1;
+       +--------------+---------------+
+       | employee_id  |   full_name   |
+       +--------------+---------------+
+       | 1            | Sheri Nowmer  |
+       +--------------+---------------+
+       1 row selected (0.129 seconds)  
+
+       0: jdbc:drill:zk=local> select [employee_id], [full_name] from cp.[employee.json]
limit 1;
+       +--------------+---------------+
+       | employee_id  |   full_name   |
+       +--------------+---------------+
+       | 1            | Sheri Nowmer  |
+       +--------------+---------------+
+       1 row selected (0.14 seconds)  
 
 ### Integer
 An integer value consists of an optional minus sign, -, followed by one or more digits.


Mime
View raw message