drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sudheesh Katkam (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (DRILL-3584) Drill Kerberos HDFS Support / Documentation
Date Mon, 13 Feb 2017 21:23:42 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862111#comment-15862111
] 

Sudheesh Katkam edited comment on DRILL-3584 at 2/13/17 9:22 PM:
-----------------------------------------------------------------

I may be wrong, but AFAIK, following Nathan's blogpost is not sufficient for Drill to authenticate
to a Kerberized HDFS (specially since ticket's expire).

The following instructions SHOULD allow for end to end (user to HDFS) authentication, but
I have not tested them since I do not have the infrastructure to setup KDC or HDFS (and not
possible to write a unit test for this, MiniDFS and Drill use the same auth library which
has static variables that MUST be different).

(1) Enable Drill login to KDC.

Once the changes in DRILL-4280 are merged (targeted for 1.10 release), the drillbit can be
started with a Kerberos principal and keytab. Add appropriate config to drill-override.conf,
something like:
{code}
drill.exec {
  security.auth.principal: “drill/_host@REALM”
  security.auth.keytab: “/etc/drill/conf/drill.keytab”
}
{code}
This assumes a service principal is created for Drill ([details here|http://www.microhowto.info/howto/create_a_service_principal_using_mit_kerberos.html]).
Ensure "drill" as a [user identity|http://hadoop.apache.org/docs/r1.2.1/hdfs_permissions_guide.html#User+Identity]
exists in HDFS (Drill uses HDFS for [Dynamic UDFs|http://drill.apache.org/docs/dynamic-udfs/],
etc.). At startup, Drill will login to KDC, and when accessing HDFS, Drill uses the configured
Kerberos credentials.

(2) Enable authentication from user to Drill.

This is strongly recommended, but optional.

(3) Enable Drill impersonation.

This is strongly recommended, but optional. Otherwise, access to HDFS happen as "drill".

(4) Enable 'secure impersonation' in HDFS.

Setup "drill" as a proxy user for HDFS with the required privileges, [details here|http://hadoop.apache.org/docs/r1.2.1/Secure_Impersonation.html].
This will allow Drill to act on behalf of the end user ("bob") when accessing HDFS. For example,
if "drill" is authorized to impersonate "bob", then while accessing the HDFS, access rights
are checked for "bob" and authentication credentials of "drill" are verified.


was (Author: sudheeshkatkam):
I may be wrong, but AFAIK, following Nathan's blogpost is not sufficient for Drill to authenticate
to a Kerberized HDFS (specially since ticket's expire).

The following instructions SHOULD allow for end to end (user to HDFS) authentication, but
I have not tested them since I do not have the infrastructure to setup KDC or HDFS (and not
possible to write a unit test for this, MiniDFS and Drill use the same auth library which
has static variables that MUST be different).

(1) Enable Drill login to KDC.

Once the changes in DRILL-4280 are merged (targeted for 1.10 release), the drillbit can be
started with a Kerberos principal and keytab. Add appropriate config to drill-override.conf,
something like:
{code}
drill.exec {
  security.auth.principal: “drill/_host@REALM”
  security.auth.keytab: “/etc/drill/conf/drill.keytab”
}
{code}
This assumes a service principal is created for Drill ([details here|http://www.microhowto.info/howto/create_a_service_principal_using_mit_kerberos.html]).
Ensure "drill" as a [user identity|http://hadoop.apache.org/docs/r1.2.1/hdfs_permissions_guide.html#User+Identity]
exists in HDFS (Drill uses HDFS for [Dynamic UDFs|http://drill.apache.org/docs/dynamic-udfs/],
etc.). At startup, Drill will login to KDC, and when accessing HDFS, Drill uses the configured
Kerberos credentials.

(2) Enable authentication from user to Drill.

This is strongly recommended, but optional.

(3) Enable Drill impersonation.

This is strongly recommended, but optional. Otherwise, access to HDFS happen as "drill".

(4) Enable secure impersonation in HDFS.

Setup "drill" as a proxy user for HDFS with the required privileges, [details here|http://hadoop.apache.org/docs/r1.2.1/Secure_Impersonation.html].
This will allow Drill to act on behalf of the end user ("bob") when accessing HDFS. For example,
if "drill" is authorized to impersonate "bob", then while accessing the HDFS, access rights
are checked for "bob" and authentication credentials of "drill" are verified.

> Drill Kerberos HDFS Support / Documentation
> -------------------------------------------
>
>                 Key: DRILL-3584
>                 URL: https://issues.apache.org/jira/browse/DRILL-3584
>             Project: Apache Drill
>          Issue Type: New Feature
>    Affects Versions: 1.1.0
>            Reporter: Hari Sekhon
>            Priority: Critical
>              Labels: security
>
> I'm trying to find Drill docs for Kerberos support for secure HDFS clusters and it doesn't
appear to be well tested / supported / documented yet.
> This product is Dead-on-Arrival if it doesn't integrate well with secure Hadoop clusters,
specifically HDFS + Kerberos (plus obviously secure kerberized Hive/HCatalog etc.)



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message