drill-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bridg...@apache.org
Subject drill git commit: edits to docs for 1.11
Date Mon, 31 Jul 2017 20:59:57 GMT
Repository: drill
Updated Branches:
  refs/heads/gh-pages 9392e3174 -> 7cab70c9f


edits to docs for 1.11


Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/7cab70c9
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/7cab70c9
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/7cab70c9

Branch: refs/heads/gh-pages
Commit: 7cab70c9fa618fabe1a217d0bb00ec84f5a442f7
Parents: 9392e31
Author: Bridget Bevens <bbevens@maprtech.com>
Authored: Mon Jul 31 13:58:53 2017 -0700
Committer: Bridget Bevens <bbevens@maprtech.com>
Committed: Mon Jul 31 13:58:53 2017 -0700

----------------------------------------------------------------------
 .../010-securing-drill-introduction.md          |  4 +-
 .../020-secure-communication-paths.md           | 15 +++----
 .../070-configuring-user-authentication.md      |  4 +-
 ...090-configuring-kerberos-auththentication.md | 43 ++++++++++++++++----
 blog/_posts/2017-07-31-drill-1.11-released.md   | 30 ++++++++------
 5 files changed, 64 insertions(+), 32 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/drill/blob/7cab70c9/_docs/configure-drill/securing-drill/010-securing-drill-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/securing-drill/010-securing-drill-introduction.md b/_docs/configure-drill/securing-drill/010-securing-drill-introduction.md
index 2701088..3d6846d 100644
--- a/_docs/configure-drill/securing-drill/010-securing-drill-introduction.md
+++ b/_docs/configure-drill/securing-drill/010-securing-drill-introduction.md
@@ -1,6 +1,6 @@
 ---
 title: "Securing Drill Introduction"
-date: 2017-03-16 01:47:58 UTC
+date: 2017-07-31 20:58:54 UTC
 parent: "Securing Drill"
 ---
 
@@ -20,7 +20,7 @@ Before connecting to a data source, you can configure Drill security features
an
 			- [Configuring User Impersonation]({{site.baseurl}}/docs/configuring-user-impersonation/)
 
 			- [Configuring Inbound Impersonation]({{site.baseurl}}/docs/configuring-inbound-impersonation/)
 
 			- [Configuring User Impersonation with Hive Authorization]({{site.baseurl}}/docs/configuring-user-impersonation-with-hive-authorization/)
 
-- **Encryption** - Drill does not support encryption as of Drill 1.10.
+- **Encryption** - Drill supports client-to-drillbit encryption in Drill 1.11.
 
 
 

http://git-wip-us.apache.org/repos/asf/drill/blob/7cab70c9/_docs/configure-drill/securing-drill/020-secure-communication-paths.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/securing-drill/020-secure-communication-paths.md b/_docs/configure-drill/securing-drill/020-secure-communication-paths.md
index dfe7f4b..4811705 100644
--- a/_docs/configure-drill/securing-drill/020-secure-communication-paths.md
+++ b/_docs/configure-drill/securing-drill/020-secure-communication-paths.md
@@ -1,6 +1,6 @@
 ---
 title: "Secure Communication Paths"
-date: 2017-03-17 22:19:49 UTC
+date: 2017-07-31 20:58:55 UTC
 parent: "Securing Drill"
 ---
 As illustrated in the following figure, Drill 1.10 features five secure communication paths.
Security features for each communication path are described their respective  sections.
@@ -25,15 +25,16 @@ The Web Console and REST API clients are web clients. Web clients can:
 ---
 **Note**
 
-Impersonation and authorization are available through the web clients only when authentication
is enabled. Otherwise, the user identity is unknown.
+Impersonation, authorization, and encryption are available through the web clients only when
authentication and encryption are enabled. Otherwise, the user identity is unknown and encryption
is not used.
 
 ---
 
-| Security Capability | Description                                                     
                                                                                         
                                                                                         
                                                                                         
      | Reference                                                            |
-|---------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------|
-| Authentication      | Users authenticate to a drillbit using a username and password form
authenticator. By default, authentication is disabled.                                   
                                                                                         
                                                                                         
   | [Configuring Web Console and REST API Security]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security)
                       |
-| Impersonation       | Drill acts on behalf of the user on the session. This is usually
the connection user (or the user that authenticates). This user can impersonate another user,
which is allowed if the connection user is authorized to impersonate the target user based
on the inbound impersonation policies (USER role).  By default, impersonation is disabled.
| [Configuring User Impersonation]({{site.baseurl}}/docs/configuring-user-impersonation/#impersonation-and-views)
and [Configuring Inbound Impersonation]({{site.baseurl}}/docs/configuring-inbound-impersonation)
|
-| Authorization       | Queries execute on behalf of the web user. Users and administrators
have different navigation bars. Various tabs are shown based on privileges. For example, only
administrators can see the Storage tab and create/read/update/delete storage plugin configuration.
                                                                                | [Configuring
Web Console and REST API Security]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security)
                       |
+| Security Capability | Description                                                     
                                                                                         
                                                                                         
                                                                                         
                                                                                         
                                       | Reference                                       
                                                                                         
                                                                      |
+|---------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Authentication      | Users authenticate to a drillbit using a username and password form
authenticator. By default, authentication is disabled.                                   
                                                                                         
                                                                                         
                                                                                         
                                    | [Configuring Web Console and REST API Security]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security)
                                                                                         
  |
+| Encryption          | Drill 1.11 supports encryption between a Drill client and Drillbit
using the Kerberos mechanism over a Java SASL framework. Encrypting the client-to-drillbit
communication pathway ensures data integrity and prevents data tampering as well as snooping.
  On the server side, enable encryption in the drill-override.conf file with the security.user.encryption.sasl.enabled
parameter. On the client side, use the sasl_encrypt parameter in the connection string. |
[Configuring Kerberos Authentication]({{site.baseurl}}/docs/configuring-kerberos-authentication/)
                                                                                         
                     |
+| Impersonation       | Drill acts on behalf of the user on the session. This is usually
the connection user (or the user that authenticates). This user can impersonate another user,
which is allowed if the connection user is authorized to impersonate the target user based
on the inbound impersonation policies (USER role).  By default, impersonation is disabled.
                                                                                         
                                 | [Configuring User Impersonation]({{site.baseurl}}/docs/configuring-user-impersonation/#impersonation-and-views)
and [Configuring Inbound Impersonation]({{site.baseurl}}/docs/configuring-inbound-impersonation)
|
+| Authorization       | Queries execute on behalf of the web user. Users and administrators
have different navigation bars. Various tabs are shown based on privileges. For example, only
administrators can see the Storage tab and create/read/update/delete storage plugin configuration.
                                                                                         
                                                                                         
                       | [Configuring Web Console and REST API Security]({{site.baseurl}}/docs/configuring-web-console-and-rest-api-security)
                                                                                         
  |          |
 
 ## Java and C++ Client to Drillbit
 

http://git-wip-us.apache.org/repos/asf/drill/blob/7cab70c9/_docs/configure-drill/securing-drill/070-configuring-user-authentication.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/securing-drill/070-configuring-user-authentication.md b/_docs/configure-drill/securing-drill/070-configuring-user-authentication.md
index 27aa9c1..fb7a6ea 100644
--- a/_docs/configure-drill/securing-drill/070-configuring-user-authentication.md
+++ b/_docs/configure-drill/securing-drill/070-configuring-user-authentication.md
@@ -1,11 +1,11 @@
 ---
 title: "Configuring User Authentication"
-date: 2017-05-17 01:38:49 UTC
+date: 2017-07-31 20:58:56 UTC
 parent: "Securing Drill"
 ---
 Authentication is the process of establishing confidence of authenticity. A Drill client
user is authenticated when a drillbit process running in a Drill cluster confirms the identity
it is presented with.  Drill 1.10 supports several authentication mechanisms through which
users can prove their identity before accessing cluster data: 
 
-* **Kerberos** - New in Drill 1.10. See [Configuring Kerberos Authentication]({{site.baseurl}}/docs/configuring-kerberos-authentication/).
+* **Kerberos** - Featuring Drill client to Drillbit encryption in Drill 1.11. See [Configuring
Kerberos Authentication]({{site.baseurl}}/docs/configuring-kerberos-authentication/).
 * **Plain** [also known as basic authentication (auth), which is username and password-based
authentication, through the Linux Pluggable Authentication Module (PAM)] - See [Configuring
Plain Authentication]({{site.baseurl}}/docs/configuring-plain-authentication/).
 * **Custom authenticators** - See [Creating Custom Authenticators]({{site.baseurl}}/docs/creating-custom-authenticators).
 

http://git-wip-us.apache.org/repos/asf/drill/blob/7cab70c9/_docs/configure-drill/securing-drill/090-configuring-kerberos-auththentication.md
----------------------------------------------------------------------
diff --git a/_docs/configure-drill/securing-drill/090-configuring-kerberos-auththentication.md
b/_docs/configure-drill/securing-drill/090-configuring-kerberos-auththentication.md
index 172146a..3261444 100644
--- a/_docs/configure-drill/securing-drill/090-configuring-kerberos-auththentication.md
+++ b/_docs/configure-drill/securing-drill/090-configuring-kerberos-auththentication.md
@@ -1,9 +1,9 @@
 ---
 title: "Configuring Kerberos Authentication"
-date: 2017-05-17 01:38:52 UTC
+date: 2017-07-31 20:58:57 UTC
 parent: "Securing Drill"
 ---
-In release 1.10 Drill supports Kerberos v5 network security authentication.  To use Kerberos
with Drill and establish connectivity, use the JDBC driver packaged with Drill 1.10.
+In release 1.11 Drill supports Kerberos v5 network security authentication and client-to-drillbit
encryption.  To use Kerberos with Drill and establish connectivity, use the JDBC driver packaged
with Drill 1.11.
 
 Kerberos allows trusted hosts to prove their identity over a network to an information system.
 A Kerberos *realm* is unique authentication domain. A centralized *key distribution center
(KDC)* coordinates authentication between a clients and servers. Clients and servers obtain
and use tickets from the KDC using a special *keytab* file to communicate with the KDC and
prove their identity to gain access to a drillbit.  Administrators must create *principal*
(user or server) identities and passwords to ensure the secure exchange of mutual authentication
information passed to and from the drillbit.   
 
@@ -14,7 +14,7 @@ See the [MIT Kerberos](http://web.mit.edu/kerberos/ "MIT Kerberos") documentatio
 
 ## Prerequisites
 
-The required Kerberos (JDBC) plugin is part of the 1.10 Drill package. To use it, you must
have a working Kerberos infrastructure, which Drill does not provide. You must be working
in a Linux-based or Windows Active Directory (AD) Kerberos environment with secure clusters
and have a Drill server configured for Kerberos. See [Enabling Authentication]({{site.baseurl}}/docs/configuring-kerberos-authentication/#enabling-authentication).
+The required Kerberos (JDBC) plugin is part of the 1.11 Drill package. To use it, you must
have a working Kerberos infrastructure, which Drill does not provide. You must be working
in a Linux-based or Windows Active Directory (AD) Kerberos environment with secure clusters
and have a Drill server configured for Kerberos. See [Enabling Authentication]({{site.baseurl}}/docs/configuring-kerberos-authentication/#enabling-authentication).
 
 ## Client Authentication Process 
 
@@ -38,9 +38,17 @@ This section provides a high-level overview of the Kerberos client authenticatio
 For Kerberos server authentication information, see the [MIT Kerberos](http://web.mit.edu/kerberos/
"MIT Kerberos") administration documentation. 
 
 
-## Enabling Authentication
+## Enabling Authentication and Encryption
 During startup, a drillbit service must authenticate. At runtime, Drill uses the keytab file.
Trust is based on the keytab file; its secrets are shared with the KDC. The drillbit service
also uses this keytab credential to validate service tickets from clients. Based on this information,
the drillbit determines whether the client’s identity can be verified to use its service.

 
+To enable encryption,set the following parameters in the `drill-override.conf` file (as shown
in the second example below): 
+
+
+- `security.user.encryption.sasl.enabled` to true. This parameter determines if the drillbit
is enabled for encryption. Only Drill 1.11 drillbits support encryption. 
+
+- `security.user.encryption.sasl.max_wrapped_size`. This parameter specifies the maximum
size of encoded buffer in bytes (maxbuffer parameter in sasl) that the client and server will
receive. Using this the SASL framework exposes maximum buffer size that the wrap function
will accept, so that Drill client/server can chop the Outbound RPC message with the size.
The maximum recommended value is 16777215. The default is 65536.
+
+
 ![kerberos client server]({{ site.baseurl }}/docs/img/kerberos-client-server.png)
  
 
@@ -97,11 +105,13 @@ During startup, a drillbit service must authenticate. At runtime, Drill
uses the
                         auth.principal:“drill/<clustername>@<REALM>.COM”,
 
                         auth.keytab:“/etc/drill/conf/drill.keytab”  
                       }  
-                security.user.auth: {
-                        enabled: true,
-                        packages += "org.apache.drill.exec.rpc.user.security",
-                        impl: "pam",
-                        pam_profiles: ["sudo", "login"]
+                security.user: {
+                        auth.enabled: true,
+                        auth.packages += "org.apache.drill.exec.rpc.user.security",
+                        auth.impl: "pam",
+                        auth.pam_profiles: ["sudo", "login"],
+						encryption.sasl.enabled: true,
+						encryption.sasl.max_wrapped_size: 65536,
                        }   
                 }
 
@@ -141,11 +151,26 @@ The following table lists configuration options for connection URLs.
See the Con
 | auth                 | Authentication mechanism. The value is deduced if not specified.
Kerberos if principal is provided. Plain  if a user and password is provided. A Drill client
can also explicitly specify a particular authentication mechanism to use using this parameter.
For example, <auth=kerberos> for Kerberos along with service_name, service_host or principal
and <auth=plain> for the Plain authentication with username and password. | Optional
          | The preference order is Kerberos and Plain.                                  
                                                                                         
                                  |
 | principal            | Drillbit service principal. The format of the principal is primary/instance@realm.
For Kerberos, the Drill service principal is derived if the value is not provided using this
configuration. service_name (primary) and service_host (instance) are used to generate a valid
principal. Since the ticket or keytab contains the realm information, the realm is optional.
                                                        | Optional           |           
                                                                                         
                                                                                         
           |
 | keytab               | For Kerberos, if the client chooses to authenticate using a keytab
rather than a ticket, set the keytab parameter to the location of the keytab file. The client
principal must be provided through the user parameter. A Kerberos ticket is used as the default
credential (It is assumed to be present on client-side. The Drill client does not generate
the required credentials.)                                              | Optional       
   |                                                                                     
                                                                                         
                           |
+| sasl_encrypt         | When set to true, ensures that a client connects to a server with
encryption capabilities. For example, Drill 1.11 drillbits, which support client-to-drillbit
encryption.                                                                              
                                                                                         
                                                                                 | Optional
          | false                                                                        
                                                                                         
                                  |
 | service_name         | Primary name of the drillbit service principal.                
                                                                                         
                                                                                         
                                                                                         
                                                                                      | Optional
          | drill                                                                        
                                                                                         
                                  |
 | service_host         | Instance name of the drillbit service principal.               
                                                                                         
                                                                                         
                                                                                         
                                                                                      | Optional
          | Since this value is usually the hostname of the node where a drillbit is running,
the default value is the drillbit hostname is  provided either through ZooKeeper or through
a direct connection string. |
 | realm                | Kerberos realm name for the drillbit service principal. The ticket
or keytab contains the realm information.                                                
                                                                                         
                                                                                         
                                                                                   | Optional
          |                                                                              
                                                                                         
                                  |
 
 
+### Client Encryption 
+A client can specify that it requires a server with encryption capabilities only by setting
the  `sasl_encrypt` connection parameter to **true**. If the cluster to which client is connecting
has encryption disabled, the client will fail to connect to that server.
+
+	drill.exec {
+ 	 security:  {
+  		  user.auth.enabled: true,
+  		  auth.mechanisms: ["KERBEROS"],
+  		  auth.principal: "drill/serverhostname@REALM.COM",
+  		  auth.keytab: "/etc/drill/conf/drill.keytab",
+  		  user.encryption.sasl.enabled: true
+ 			  }
+	}
+
+
 ### Connection URL Examples
 
 The following five examples show the JDBC connection URL that the embedded JDBC client uses
for Kerberos authentication. The first section, Example of a Simple Connection URL, includes
a simple connection string and the second section, Examples of Connection URLs Used with Previously
Generated TGTs, includes examples to use with previously generated TGTs.

http://git-wip-us.apache.org/repos/asf/drill/blob/7cab70c9/blog/_posts/2017-07-31-drill-1.11-released.md
----------------------------------------------------------------------
diff --git a/blog/_posts/2017-07-31-drill-1.11-released.md b/blog/_posts/2017-07-31-drill-1.11-released.md
index c379ff4..3628bd1 100644
--- a/blog/_posts/2017-07-31-drill-1.11-released.md
+++ b/blog/_posts/2017-07-31-drill-1.11-released.md
@@ -12,16 +12,16 @@ The release provides the following bug fixes and improvements:
 
 ## Cryptography-Related Functions (DRILL-5634)
 Drill provides the following cryptographic-related functions:
-
-- aes_encrypt()
-- aes_decrypt()
-- md5()
-- sha()
-- sha1()
-- sha2()  
-
+
+- aes_encrypt()
+- aes_decrypt()
+- md5()
+- sha()
+- sha1()
+- sha2()  
+
 ## Spill to Disk for Hash Aggregate Operator (DRILL-5457)  
-The Hash aggregate operator can spill data to disk in cases where the operation exceeds the
set memory limit.   
+The Hash aggregate operator can spill data to disk in cases where the operation exceeds the
set memory limit. Note that you may need to increase the default value of the `planner.memory.max_query_memory_per_node`
option due to insufficient memory.      
 
 ## Format Plugin Support for PCAP Files (DRILL-5432)  
 A “pcap” format plugin enables Drill to read PCAP files. You must add the “pcap”
format to the dfs storage plugin configuration, as shown:  
@@ -40,7 +40,7 @@ The `store.parquet.writer.use_single_fs_block` option enables Drill to write
a P
 The `drill.exec.profiles.store.inmemory` option enables Drill to store query profiles in
memory instead of writing the query profiles to disk. The `drill.exec.profiles.store.capacity`
option sets the maximum number of most recent profiles to retain in memory.  
 
 ## Configurable CTAS Directory and File Permissions Option (DRILL-5391)  
-You can use the `exec.persistent_table.umask` configuration option, at the system or session
level, to modify permissions on directories and files that result from running the CTAS command.
By default, the option is set to 002, which sets the default directory permissions to 775
and default file permissions to -664.   
+You can use the `exec.persistent_table.umask` configuration option, at the system or session
level, to modify permissions on directories and files that result from running the CTAS command.
By default, the option is set to 002, which sets the default directory permissions to 775
and default file permissions to 664.   
 
 ## Support for Network Encryption (DRILL-4335)  
 Drill can use SASL to support network encryption between the Drill client and drillbits,
and also between drillbits.  
@@ -48,8 +48,14 @@ Drill can use SASL to support network encryption between the Drill client
and dr
 ## Metadata file Stores Relative Paths (DRILL-3867)  
 Drill now stores the relative path in the metadata file (versus the absolute path), which
enables you to move partitioned Parquet directories from one location in DFS to another without
having to rebuild the Parquet metadata files; the metadata remains valid in the new location.
 
 
-## Support for ANSI_QUOTES (DRILL-3510)  
-In addition to back ticks, the SQL parser in Drill can use double quotes as identifier quotes.
Use the `planner.parser.quoting_identifiers` configuration option, at the system or session
level, to set the type of identifier quotes that the SQL parser in Drill uses.  
+## Support for Additional Quoting Identifiers (DRILL-3510)  
+In addition to back ticks, the SQL parser in Drill can use double quotes and square brackets
as identifier quotes. Use the `planner.parser.quoting_identifiers` configuration option, at
the system or session level, to set the type of identifier quotes that the SQL parser in Drill
uses, as shown:  
+
+       ALTER SESSION SET planner.parser.quoting_identifiers = '"';  
+       ALTER SESSION SET planner.parser.quoting_identifiers = '[';  
+       ALTER SESSION SET planner.parser.quoting_identifiers = '`';  
+
+The default setting is back ticks. The quoting identifier used in queries must match the
setting. If you use another type of quoting identifier, Drill returns an error.  
 
 You can find a complete list of JIRAs resolved in the 1.11.0 release [here](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313820&version=12339943).
 


Mime
View raw message