hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kyle Leckie <kylec...@microsoft.com>
Subject RE: Hadoop Summit: Security Design Lounge Session
Date Thu, 04 Jul 2013 00:20:45 GMT
Thanks for the excellent summary Larry,

Questions for the group:
I have taken a quick look at how pluggable token validation could be added to the RPC endpoints:
	- Are there any current approaches that I should examined before I continue with my investigation?
	- For server Auth; I would like to consider TLS. Has there been any benchmarking of a well
implemented server stack (supports session caching and has algorithms configured for performance)?

-----Original Message-----
From: Larry McCay [mailto:lmccay@hortonworks.com] 
Sent: Monday, July 1, 2013 2:40 PM
To: common-dev@hadoop.apache.org
Subject: Hadoop Summit: Security Design Lounge Session

All -

Last week at Hadoop Summit there was a room dedicated as the summit Design Lounge.
This was a place where like folks could get together and talk about design issues with other
contributors with a simple flip board and some beanbag chairs.
We used this as an opportunity to bootstrap some discussions within common-dev for security
related topics. I'd like to summarize the security session and takeaways here for everyone.

This summary and set of takeaways are largely from memory. 
Please - anyone that attended - feel free to correct anything that is inaccurate or omitted.

Pretty well attended - companies represented:
* Yahoo!
* Microsoft
* Hortonworks
* Cloudera
* Intel
* eBay
* Voltage Security
* Flying Penguins
* others...

Most folks were pretty engaged throughout the session.
We set expectations as a meet and greet/project kickoff - project being the emerging security
development community.

In order to keep the scope of conversations manageable we tried to keep focused on authentication
and the ideas around SSO and tokens.

We discussed kerberos as:
1. major pain point and barrier to entry for some 2. seemingly perfect for others
	a. obviously requiring backward compatibility

It seemed to be consensus that:
1. user authentication should be easily integrated with alternative enterprise identity solutions
2. that service identity issues should not require thousands of service identities added to
enterprise user repositories 3. that customers should not be forced to install/deploy and
manage a KDC for services - this implies a couple options:
	a. alternatives to kerberos for service identities
	b. hadoop KDC implementation - ie. ApacheDS?

There was active discussion around:
1. Hadoop SSO server
	a. acknowledgement of Hadoop SSO tokens as something that can be standardized for representing
both the identity and authentication event data as well and access tokens representing a verifiable
means for the authenticated identity to access resources or services
	b. a general understanding of Hadoop SSO as being an analogue and alternative for the kerberos
KDC and the related tokens being analogous to TGTs and service tickets
	c. an agreement that there are interesting attributes about the authentication event that
may be useful in cross cluster trust for SSO - such as a rating of authentication strength
and number of factors, etc
	d. that existing Hadoop tokens - ie. delegation, job, block access - will all continue to
work and that we are initially looking at alternatives to the KDC, TGTs and service tickets
2. authentication mechanism discovery by clients - Daryn Sharp has done a bunch of work around
this and our SSO solution may want to consider a similar mechanism for discovering trusted
IDPs and service endpoints 3. backward compatibility - kerberos shops need to just continue
to work 4. some insight into where/how folks believe that token based authentication can be
accomplished within existing contracts - SASL/GSSAPI, REST, web ui 5. what the establishment
of a cross cutting concern community around security and what that means in terms of the Apache
way - email lists, wiki, Jiras across projects, etc 6. dependencies, rolling updates, patching
and how it related to hadoop projects versus packaging 7. collaboration road ahead

A number of breakout discussions were had outside of the designated design lounge session
as well.

Takeaways for the immediate road ahead:
1. common-dev may be sufficient to discuss security related topics
	a. many developers are already subscribed to it
	b. there is not that much traffic there anyway
	c. we can discuss a more security focused list if we like 2. we will discuss the establishment
of a wiki space for a holistic view of security model, patterns, approaches, etc 3. we will
begin discussion on common-dev in near-term for the following:
	a. discuss and agree on the high level moving parts required for our goals for authentication:
SSO service, tokens, token validation handlers, credential management tools, etc
	b. discuss and agree on the natural seams across these moving parts and agree on collaboration
by tackling various pieces in a divide and conquer approach
	c. more than likely - the first piece that will need some immediate discussion will be the
shape and form of the tokens
	d. we will follow up or supplement discussions with POC code patches and/or specs attached
to jiras

Overall, design lounge was rather effective for what we wanted to do - which was to bootstrap
discussions and collaboration within the community at large. As always, no specific decisions
have been made during this session and we can discuss any or all of this within common-dev
and on related jiras.

Jiras related to the security development group and these discussions:

Centralized SSO/Token Server https://issues.apache.org/jira/browse/HADOOP-9533
Token based authentication and SSO https://issues.apache.org/jira/browse/HADOOP-9392
Document/analyze current Hadoop security model https://issues.apache.org/jira/browse/HADOOP-9621
Improve Hadoop security - Use cases https://issues.apache.org/jira/browse/HADOOP-9671



View raw message