directory-kerby mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li, Jiajia" <jiajia...@intel.com>
Subject RE: [DISCUSS] Merge HAS to Apache Kerby
Date Tue, 28 Nov 2017 02:16:54 GMT
Thanks Colm.

> It sounds like a really interesting project. 
I'm glad to here that.

> Have you got any feedback from the Hadoop project about it?
We haven't proposed this solution in the hadoop community.

> I'm finding it hard to understand exactly how it works though based on the README. Could
you describe how it works from a really basic point of view for say a simple Hadoop client?
Normally I just have to use "kinit" to get a kerberos ticket and then I am authenticated to
invoke on HDFS. How does HAS work differently? Where does the token pre-auth stuff fit in?

Following are the steps of user accessing HDFS service, taking the cmd "hadoop fs -ls /" as
an example:
1. user runs the command "hadoop fs -ls /"
2. Hadoop client will call the "HasLoginModule", https://github.com/apache/directory-kerby/blob/has-project/has/has-client/src/main/java/org/apache/hadoop/has/client/HasLoginModule.java
3. "HasLoginModule" will call the "HasClient", https://github.com/apache/directory-kerby/blob/438904f7e557a085c8c336efd2d2be304291d246/has/has-client/src/main/java/org/apache/hadoop/has/client/HasLoginModule.java#L237
4. "HasClient" will get the plugin type from config, then choose the right client plugin,
the client plugin will collect and add some user info to "AuthToken", the following is the
client plugin interface:

// Get the login module type ID, used to distinguish this module from others. 
// Should correspond to the server side module.
String getLoginType()

// Perform all the client side login logics, the results wrapped in an AuthToken, 
// will be validated by HAS server.
AuthToken login(Conf loginConf) throws HasLoginException

5. Then "HasClient" sends the "AuthToken" to HAS Server through HTTPS;
6. After HAS server receives the message, it will call the server plugin, server plugin will
verify the user info in AuthToken, the following is the server plugin interface:

// Get the login module type ID, used to distinguish this module from others. 
// Should correspond to the client side module.
String getLoginType()

// Perform all the server side authentication logics, the results wrapped in an "AuthToken",

// will be used to exchange a Kerberos ticket.
AuthToken authenticate(AuthToken userToken) throws HasAuthenException

7. If the user info is verified in existing user authentication system, server plugin will
return the verified "AuthToken" to Kerby KDC
8. Kerby KDC will issue the TGT ticket using the TokenPreauth, then send the TGT to HasClient
through HTTPS
9. Now user login successful, could continue the others steps, such as: getting SGT ticket.

We replace the step through "kinit" to get Kerberos Ticket. There are two important benefits:
1. The user's principal may not be in the backend, security admins won't have to migrate and
sync up their user accounts to Kerberos back and forth.
2. Multiple users could run the job at the same time and in the same machine, through collecting
user info from environment variables in step4.


Thanks,
Jiajia

-----Original Message-----
From: Colm O hEigeartaigh [mailto:coheigea@apache.org] 
Sent: Monday, November 27, 2017 6:54 PM
To: kerby@directory.apache.org
Cc: Apache Directory Developers List <dev@directory.apache.org>
Subject: Re: [DISCUSS] Merge HAS to Apache Kerby

Hi Jiajia,

It sounds like a really interesting project. Have you got any feedback from the Hadoop project
about it?

I'm finding it hard to understand exactly how it works though based on the README. Could you
describe how it works from a really basic point of view for say a simple Hadoop client? Normally
I just have to use "kinit" to get a kerberos ticket and then I am authenticated to invoke
on HDFS. How does HAS work differently? Where does the token pre-auth stuff fit in?

Colm.


On Fri, Nov 24, 2017 at 3:30 AM, Li, Jiajia <jiajia.li@intel.com> wrote:

> Hi all,
>
> I would like to post a proposal about merging a new project HAS 
> (Hadoop Authentication Service) to Apache Kerby. HAS is led by Intel 
> and Alibaba, it is a solution to support the authentication of open 
> source big data ecosystem in cloud computing platforms. I've created a 
> new branch "has-project" in Kerby, HAS is under "has" folder. Please 
> look at https://github.com/apache/directory-kerby/tree/has-project/has 
> for details.
>
> Background and motivation:
> At present, the open source big data ecosystems (Hadoop/Spark) only 
> has the built-in Kerberos support on the security authentication. HAS 
> aims to build a standalone authentication service for the big data 
> ecosystem that simplifies the support of Kerberos and allows to use 
> more authentication methods.
>
> Targets users:
> HAS supports various authentication mechanisms other than just 
> Kerberos, and it provides a new authentication mechanism can be easy 
> customized and plugin with existing user authentication and 
> authorization system, and security admins won't have to migrate and 
> sync up their user accounts to Kerberos back and forth.
>
> Architecture & Design:
> HAS provides a new authentication mechanism ("Kerberos-based token 
> authentication"), depending on the "TokenPreauth" provided by Apache Kerby.
> Please look at 
> https://github.com/apache/directory-kerby/blob/has-project/
> has/README.md for details.
>
> Features:
> 1.      Provides new authentication mechanism plugin APIs to customize and
> plugin with existing user authentication and authorization system. 
> Please look at 
> https://github.com/apache/directory-kerby/blob/has-project/
> has/README.md for details.
> 2.      Provides lots of REST APIs and facility tools to simplify the
> support of Kerberos. Kerberos is essentially a protocol, or secure 
> channel, doesn't have to be that complex to users. Please look at 
> https://github.com/apache/directory-kerby/blob/has-project/
> has/doc/rest-api.md for details.
> 3.      Provides MySQL backend for High Availability. Please look at
> https://github.com/apache/directory-kerby/blob/has-project/
> has/doc/mysql-backend.md for details.
> 4.      New authentication mechanism now supports most of the components
> of open source big data ecosystem with little or no changes to 
> components, including HDFS, HBase, Zookeeper, Hive, Spark.... Please 
> look at 
> https://github.com/apache/directory-kerby/tree/has-project/has/support
> s
> for details.
>
> Practice
> This solution has been deployed in Alibaba Cloud E-MapReduce production.
>
> Why to merge?
> HAS provides a complete Hadoop/Spark authentication framework and 
> solution based on Kerberos, HAS can help to upgrade Kerby KDC, make it 
> more solid and stronger. And if HAS can be merged to Apache Kerby, 
> community will help HAS grow faster and users can more easily using 
> this solution in their own production. We have two suggestions about how to merge:
> - Option1:
> Create a standalone module "kerby-has", putting HAS project under this 
> module.
> - Option2:
> Suggest replacing kerby-kdc module with HAS, upgrade the Kerby KDC.
>
> Contributors:
> Jiajia, Li (Intel)
> Lin, Zeng (Intel)
> Zhiqiang, Zhang (Intel)
> Kai, Zheng (Intel)
> Wei, Wu (Alibaba)
> Jun, Song (Alibaba)
> Long, Cao (Alibaba)
> Zhenyuan, Wei (Alibaba)
>
> Your review efforts are truly appreciated, please feel free to provide 
> us your feedback.
>
> Regards,
> Jiajia
>
>
>
>


--
Colm O hEigeartaigh

Talend Community Coder
http://coders.talend.com
Mime
View raw message