hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alejandro Abdelnur <t...@cloudera.com>
Subject Re: [DISCUSS] Hadoop SSO/Token Server Components
Date Tue, 06 Aug 2013 20:15:56 GMT
yep, that is what I meant. Thanks Chris


On Tue, Aug 6, 2013 at 1:12 PM, Chris Nauroth <cnauroth@hortonworks.com>wrote:

> Perhaps this is also a good opportunity to try out the new "branch
> committers" clause in the bylaws, enabling non-committers who are working
> on this to commit to the feature branch.
>
>
> http://mail-archives.apache.org/mod_mbox/hadoop-general/201308.mbox/%3CCACO5Y4we4d8knB_xU3a=hr2gbeQO5m3vaU+inbA0Li1i9e21DQ@mail.gmail.com%3E
>
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
>
>
>
> On Tue, Aug 6, 2013 at 1:04 PM, Alejandro Abdelnur <tucu@cloudera.com
> >wrote:
>
> > Larry,
> >
> > Sorry for the delay answering. Thanks for laying down things, yes, it
> makes
> > sense.
> >
> > Given the large scope of the changes, number of JIRAs and number of
> > developers involved, wouldn't make sense to create a feature branch for
> all
> > this work not to destabilize (more ;) trunk?
> >
> > Thanks again.
> >
> >
> > On Tue, Jul 30, 2013 at 9:43 AM, Larry McCay <lmccay@hortonworks.com>
> > wrote:
> >
> > > The following JIRA was filed to provide a token and basic authority
> > > implementation for this effort:
> > > https://issues.apache.org/jira/browse/HADOOP-9781
> > >
> > > I have attached an initial patch though have yet to submit it as one
> > since
> > > it is dependent on the patch for CMF that was posted to:
> > > https://issues.apache.org/jira/browse/HADOOP-9534
> > > and this patch still has a couple outstanding issues - javac warnings
> for
> > > com.sun classes for certification generation and 11 javadoc warnings.
> > >
> > > Please feel free to review the patches and raise any questions or
> > concerns
> > > related to them.
> > >
> > > On Jul 26, 2013, at 8:59 PM, Larry McCay <lmccay@hortonworks.com>
> wrote:
> > >
> > > > Hello All -
> > > >
> > > > In an effort to scope an initial iteration that provides value to the
> > > community while focusing on the pluggable authentication aspects, I've
> > > written a description for "Iteration 1". It identifies the goal of the
> > > iteration, the endstate and a set of initial usecases. It also
> enumerates
> > > the components that are required for each usecase. There is a scope
> > section
> > > that details specific things that should be kept out of the first
> > > iteration. This is certainly up for discussion. There may be some of
> > these
> > > things that can be contributed in short order. If we can add some
> things
> > in
> > > without unnecessary complexity for the identified usecases then we
> > should.
> > > >
> > > > @Alejandro - please review this and see whether it satisfies your
> point
> > > for a definition of what we are building.
> > > >
> > > > In addition to the document that I will paste here as text and
> attach a
> > > pdf version, we have a couple patches for components that are
> identified
> > in
> > > the document.
> > > > Specifically, COMP-7 and COMP-8.
> > > >
> > > > I will be posting COMP-8 patch to the HADOOP-9534 JIRA which was
> filed
> > > specifically for that functionality.
> > > > COMP-7 is a small set of classes to introduce JsonWebToken as the
> token
> > > format and a basic JsonWebTokenAuthority that can issue and verify
> these
> > > tokens.
> > > >
> > > > Since there is no JIRA for this yet, I will likely file a new JIRA
> for
> > a
> > > SSO token implementation.
> > > >
> > > > Both of these patches assume to be modules within
> > > hadoop-common/hadoop-common-project.
> > > > While they are relatively small, I think that they will be pulled in
> by
> > > other modules such as hadoop-auth which would likely not want a
> > dependency
> > > on something larger like
> > hadoop-common/hadoop-common-project/hadoop-common.
> > > >
> > > > This is certainly something that we should discuss within the
> community
> > > for this effort though - that being, exactly how to add these libraries
> > so
> > > that they are most easily consumed by existing projects.
> > > >
> > > > Anyway, the following is the Iteration-1 document - it is also
> attached
> > > as a pdf:
> > > >
> > > > Iteration 1: Pluggable User Authentication and Federation
> > > >
> > > > Introduction
> > > > The intent of this effort is to bootstrap the development of
> pluggable
> > > token-based authentication mechanisms to support certain goals of
> > > enterprise authentication integrations. By restricting the scope of
> this
> > > effort, we hope to provide immediate benefit to the community while
> > keeping
> > > the initial contribution to a manageable size that can be easily
> > reviewed,
> > > understood and extended with further development through follow up
> JIRAs
> > > and related iterations.
> > > >
> > > > Iteration Endstate
> > > > Once complete, this effort will have extended the authentication
> > > mechanisms - for all client types - from the existing: Simple, Kerberos
> > and
> > > Plain (for RPC) to include LDAP authentication and SAML based
> federation.
> > > In addition, the ability to provide additional/custom authentication
> > > mechanisms will be enabled for users to plug in their preferred
> > mechanisms.
> > > >
> > > > Project Scope
> > > > The scope of this effort is a subset of the features covered by the
> > > overviews of HADOOP-9392 and HADOOP-9533. This effort concentrates on
> > > enabling Hadoop to issue, accept/validate SSO tokens of its own. The
> > > pluggable authentication mechanism within SASL/RPC layer and the
> > > authentication filter pluggability for REST and UI components will be
> > > leveraged and extended to support the results of this effort.
> > > >
> > > > Out of Scope
> > > > In order to scope the initial deliverable as the minimally viable
> > > product, a handful of things have been simplified or left out of scope
> > for
> > > this effort. This is not meant to say that these aspects are not useful
> > or
> > > not needed but that they are not necessary for this iteration. We do
> > > however need to ensure that we don’t do anything to preclude adding
> them
> > in
> > > future iterations.
> > > > 1. Additional Attributes - the result of authentication will continue
> > to
> > > use the existing hadoop tokens and identity representations. Additional
> > > attributes used for finer grained authorization decisions will be added
> > > through follow-up efforts.
> > > > 2. Token revocation - the ability to revoke issued identity tokens
> will
> > > be added later
> > > > 3. Multi-factor authentication - this will likely require additional
> > > attributes and is not necessary for this iteration.
> > > > 4. Authorization changes - we will require additional attributes for
> > the
> > > fine-grained access control plans. This is not needed for this
> iteration.
> > > > 5. Domains - we assume a single flat domain for all users
> > > > 6. Kinit alternative - we can leverage existing REST clients such as
> > > cURL to retrieve tokens through authentication and federation for the
> > time
> > > being
> > > > 7. A specific authentication framework isn’t really necessary within
> > the
> > > REST endpoints for this iteration. If one is available then we can use
> it
> > > otherwise we can leverage existing things like Apache Shiro within a
> > > servlet filter.
> > > >
> > > > In Scope
> > > > What is in scope for this effort is defined by the usecases described
> > > below. Components required for supporting the usecases are summarized
> for
> > > each client type. Each component is a candidate for a JIRA subtask -
> > though
> > > multiple components are likely to be included in a JIRA to represent a
> > set
> > > of functionality rather than individual JIRAs per component.
> > > >
> > > > Terminology and Naming
> > > > The terms and names of components within this document are merely
> > > descriptive of the functionality that they represent. Any similarity or
> > > difference in names or terms from those that are found in other
> documents
> > > are not intended to make any statement about those other documents or
> the
> > > descriptions within. This document represents the pluggable
> > authentication
> > > mechanisms and server functionality required to replace Kerberos.
> > > >
> > > > Ultimately, the naming of the implementation classes will be a
> product
> > > of the patches accepted by the community.
> > > >
> > > > Usecases:
> > > > client types: REST, CLI, UI
> > > > authentication types: Simple, Kerberos, authentication/LDAP,
> > > federation/SAML
> > > >
> > > > Simple and Kerberos
> > > > Simple and Kerberos usecases continue to work as they do today. The
> > > addition of Authentication/LDAP and Federation/SAML are added through
> the
> > > existing pluggability points either as they are or with required
> > extension.
> > > Either way, continued support for Simple and Kerberos must not require
> > > changes to existing deployments in the field as a result of this
> effort.
> > > >
> > > > REST
> > > > USECASE REST-1 Authentication/LDAP:
> > > > For REST clients, we will provide the ability to:
> > > > 1. use cURL to Authenticate via LDAP through an IdP endpoint exposed
> by
> > > an AuthenticationServer instance via REST calls to:
> > > >    a. authenticate - passing username/password returning a hadoop
> > > id_token
> > > >    b. get-access-token - from the TokenGrantingService by passing the
> > > hadoop id_token as an Authorization: Bearer token along with the
> desired
> > > service name (master service name) returning a hadoop access token
> > > > 2. Successfully invoke a hadoop service REST API passing the hadoop
> > > access token through an HTTP header as an Authorization Bearer token
> > > >    a. validation of the incoming token on the service endpoint is
> > > accomplished by an SSOAuthenticationHandler
> > > > 3. Successfully block access to a REST resource when presenting a
> > hadoop
> > > access token intended for a different service
> > > >    a. validation of the incoming token on the service endpoint is
> > > accomplished by an SSOAuthenticationHandler
> > > >
> > > > USECASE REST-2 Federation/SAML:
> > > > We will also provide federation capabilities for REST clients such
> > that:
> > > > 1. acquire SAML assertion token from a trusted IdP (shibboleth?) and
> > > persist in a permissions protected file - ie.
> ~/.hadoop_tokens/.idp_token
> > > > 2. use cURL to Federate a token from a trusted IdP through an SP
> > > endpoint exposed by an AuthenticationServer(FederationServer?) instance
> > via
> > > REST calls to:
> > > >    a. federate - passing a SAML assertion as an Authorization: Bearer
> > > token returning a hadoop id_token
> > > >       - can copy and paste from commandline or use cat to include
> > > persisted token through "--Header Authorization: Bearer 'cat
> > > ~/.hadoop_tokens/.id_token'"
> > > >    b. get-access-token - from the TokenGrantingService by passing the
> > > hadoop id_token as an Authorization: Bearer token along with the
> desired
> > > service name (master service name) to the TokenGrantingService
> returning
> > a
> > > hadoop access token
> > > > 3. Successfully invoke a hadoop service REST API passing the hadoop
> > > access token through an HTTP header as an Authorization Bearer token
> > > >    a. validation of the incoming token on the service endpoint is
> > > accomplished by an SSOAuthenticationHandler
> > > > 4. Successfully block access to a REST resource when presenting a
> > hadoop
> > > access token intended for a different service
> > > >    a. validation of the incoming token on the service endpoint is
> > > accomplished by an SSOAuthenticationHandler
> > > >
> > > > REQUIRED COMPONENTS for REST USECASES:
> > > > COMP-1. REST client - cURL or similar
> > > > COMP-2. REST endpoint for BASIC authentication to LDAP - IdP endpoint
> > > example - returning hadoop id_token
> > > > COMP-3. REST endpoint for federation with SAML Bearer token -
> > shibboleth
> > > SP?|OpenSAML? - returning hadoop id_token
> > > > COMP-4. REST TokenGrantingServer endpoint for acquiring hadoop access
> > > tokens from hadoop id_tokens
> > > > COMP-5. SSOAuthenticationHandler to validate incoming hadoop access
> > > tokens
> > > > COMP-6. some source of a SAML assertion - shibboleth IdP?
> > > > COMP-7. hadoop token and authority implementations
> > > > COMP-8. core services for crypto support for signing, verifying and
> PKI
> > > management
> > > >
> > > > CLI
> > > > USECASE CLI-1 Authentication/LDAP:
> > > > For CLI/RPC clients, we will provide the ability to:
> > > > 1. use cURL to Authenticate via LDAP through an IdP endpoint exposed
> by
> > > an AuthenticationServer instance via REST calls to:
> > > >    a. authenticate - passing username/password returning a hadoop
> > > id_token
> > > >       - for RPC clients we need to persist the returned hadoop
> identity
> > > token in a file protected by fs permissions so that it may be leveraged
> > > until expiry
> > > >       - directing the returned response to a file may suffice for now
> > > something like ">~/.hadoop_tokens/.id_token"
> > > > 2. use hadoop CLI to invoke RPC API on a specific hadoop service
> > > >    a. RPC client negotiates a TokenAuth method through SASL layer,
> > > hadoop id_token is retrieved from ~/.hadoop_tokens/.id_token is passed
> as
> > > Authorization: Bearer token to the get-access-token REST endpoint
> exposed
> > > by TokenGrantingService returning a hadoop access token
> > > >    b. RPC server side validates the presented hadoop access token and
> > > continues to serve request
> > > >    c. Successfully invoke a hadoop service RPC API
> > > >
> > > > USECASE CLI-2 Federation/SAML:
> > > > For CLI/RPC clients, we will provide the ability to:
> > > > 1. acquire SAML assertion token from a trusted IdP (shibboleth?) and
> > > persist in a permissions protected file - ie.
> ~/.hadoop_tokens/.idp_token
> > > > 2. use cURL to Federate a token from a trusted IdP through an SP
> > > endpoint exposed by an AuthenticationServer(FederationServer?) instance
> > via
> > > REST calls to:
> > > >    a. federate - passing a SAML assertion as an Authorization: Bearer
> > > token returning a hadoop id_token
> > > >       - can copy and paste from commandline or use cat to include
> > > previously persisted token through "--Header Authorization: Bearer 'cat
> > > ~/.hadoop_tokens/.id_token'"
> > > > 3. use hadoop CLI to invoke RPC API on a specific hadoop service
> > > >    a. RPC client negotiates a TokenAuth method through SASL layer,
> > > hadoop id_token is retrieved from ~/.hadoop_tokens/.id_token is passed
> as
> > > Authorization: Bearer token to the get-access-token REST endpoint
> exposed
> > > by TokenGrantingService returning a hadoop access token
> > > >    b. RPC server side validates the presented hadoop access token and
> > > continues to serve request
> > > >    c. Successfully invoke a hadoop service RPC API
> > > >
> > > > REQUIRED COMPONENTS for CLI USECASES - (beyond those required for
> > REST):
> > > > COMP-9. TokenAuth Method negotiation, etc
> > > > COMP-10. Client side implementation to leverage REST endpoint for
> > > acquiring hadoop access tokens given a hadoop id_token
> > > > COMP-11. Server side implementation to validate incoming hadoop
> access
> > > tokens
> > > >
> > > > UI
> > > > Various Hadoop services have their own web UI consoles for
> > > administration and end user interactions. These consoles need to also
> > > benefit from the pluggability of authentication mechansims to be on par
> > > with the access control of the cluster REST and RPC APIs.
> > > > Web consoles are protected with an WebSSOAuthenticationHandler which
> > > will be configured for either authentication or federation.
> > > >
> > > > USECASE UI-1 Authentication/LDAP:
> > > > For the authentication usecase:
> > > > 1. User’s browser requests access to a UI console page
> > > > 2. WebSSOAuthenticationHandler intercepts the request and redirects
> the
> > > browser to an IdP web endpoint exposed by the AuthenticationServer
> > passing
> > > the requested url as the redirect_url
> > > > 3. IdP web endpoint presents the user with a FORM over https
> > > >    a. user provides username/password and submits the FORM
> > > > 4. AuthenticationServer authenticates the user with provided
> > credentials
> > > against the configured LDAP server and:
> > > >    a. leverages a servlet filter or other authentication mechanism
> for
> > > the endpoint and authenticates the user with a simple LDAP bind with
> > > username and password
> > > >    b. acquires a hadoop id_token and uses it to acquire the required
> > > hadoop access token which is added as a cookie
> > > >    c. redirects the browser to the original service UI resource via
> the
> > > provided redirect_url
> > > > 5. WebSSOAuthenticationHandler for the original UI resource
> > interrogates
> > > the incoming request again for an authcookie that contains an access
> > token
> > > upon finding one:
> > > >    a. validates the incoming token
> > > >    b. returns the AuthenticationToken as per AuthenticationHandler
> > > contract
> > > >    c. AuthenticationFilter adds the hadoop auth cookie with the
> > expected
> > > token
> > > >    d. serves requested resource for valid tokens
> > > >    e. subsequent requests are handled by the AuthenticationFilter
> > > recognition of the hadoop auth cookie
> > > >
> > > > USECASE UI-2 Federation/SAML:
> > > > For the federation usecase:
> > > > 1. User’s browser requests access to a UI console page
> > > > 2. WebSSOAuthenticationHandler intercepts the request and redirects
> the
> > > browser to an SP web endpoint exposed by the AuthenticationServer
> passing
> > > the requested url as the redirect_url. This endpoint:
> > > >    a. is dedicated to redirecting to the external IdP passing the
> > > required parameters which may include a redirect_url back to itself as
> > well
> > > as encoding the original redirect_url so that it can determine it on
> the
> > > way back to the client
> > > > 3. the IdP:
> > > >    a. challenges the user for credentials and authenticates the user
> > > >    b. creates appropriate token/cookie and redirects back to the
> > > AuthenticationServer endpoint
> > > > 4. AuthenticationServer endpoint:
> > > >    a. extracts the expected token/cookie from the incoming request
> and
> > > validates it
> > > >    b. creates a hadoop id_token
> > > >    c. acquires a hadoop access token for the id_token
> > > >    d. creates appropriate cookie and redirects back to the original
> > > redirect_url - being the requested resource
> > > > 5. WebSSOAuthenticationHandler for the original UI resource
> > interrogates
> > > the incoming request again for an authcookie that contains an access
> > token
> > > upon finding one:
> > > >    a. validates the incoming token
> > > >    b. returns the AuthenticationToken as per AuthenticationHandler
> > > contrac
> > > >    c. AuthenticationFilter adds the hadoop auth cookie with the
> > expected
> > > token
> > > >    d. serves requested resource for valid tokens
> > > >    e. subsequent requests are handled by the AuthenticationFilter
> > > recognition of the hadoop auth cookie
> > > > REQUIRED COMPONENTS for UI USECASES:
> > > > COMP-12. WebSSOAuthenticationHandler
> > > > COMP-13. IdP Web Endpoint within AuthenticationServer for FORM based
> > > login
> > > > COMP-14. SP Web Endpoint within AuthenticationServer for 3rd party
> > token
> > > federation
> > > >
> > > >
> > > >
> > > > On Wed, Jul 10, 2013 at 1:59 PM, Brian Swan <
> Brian.Swan@microsoft.com>
> > > wrote:
> > > > Thanks, Larry. That is what I was trying to say, but you've said it
> > > better and in more detail. :-) To extract from what you are saying: "If
> > we
> > > were to reframe the immediate scope to the lowest common denominator of
> > > what is needed for accepting tokens in authentication plugins then we
> > > gain... an end-state for the lowest common denominator that enables
> code
> > > patches in the near-term is the best of both worlds."
> > > >
> > > > -Brian
> > > >
> > > > -----Original Message-----
> > > > From: Larry McCay [mailto:lmccay@hortonworks.com]
> > > > Sent: Wednesday, July 10, 2013 10:40 AM
> > > > To: common-dev@hadoop.apache.org
> > > > Cc: daryn@yahoo-inc.com; Kai Zheng; Alejandro Abdelnur
> > > > Subject: Re: [DISCUSS] Hadoop SSO/Token Server Components
> > > >
> > > > It seems to me that we can have the best of both worlds here...it's
> all
> > > about the scoping.
> > > >
> > > > If we were to reframe the immediate scope to the lowest common
> > > denominator of what is needed for accepting tokens in authentication
> > > plugins then we gain:
> > > >
> > > > 1. a very manageable scope to define and agree upon 2. a deliverable
> > > that should be useful in and of itself 3. a foundation for community
> > > collaboration that we build on for higher level solutions built on this
> > > lowest common denominator and experience as a working community
> > > >
> > > > So, to Alejandro's point, perhaps we need to define what would make
> #2
> > > above true - this could serve as the "what" we are building instead of
> > the
> > > "how" to build it.
> > > > Including:
> > > > a. project structure within hadoop-common-project/common-security or
> > the
> > > like b. the usecases that would need to be enabled to make it a self
> > > contained and useful contribution - without higher level solutions c.
> the
> > > JIRA/s for contributing patches d. what specific patches will be needed
> > to
> > > accomplished the usecases in #b
> > > >
> > > > In other words, an end-state for the lowest common denominator that
> > > enables code patches in the near-term is the best of both worlds.
> > > >
> > > > I think this may be a good way to bootstrap the collaboration process
> > > for our emerging security community rather than trying to tackle a huge
> > > vision all at once.
> > > >
> > > > @Alejandro - if you have something else in mind that would bootstrap
> > > this process - that would great - please advise.
> > > >
> > > > thoughts?
> > > >
> > > > On Jul 10, 2013, at 1:06 PM, Brian Swan <Brian.Swan@microsoft.com>
> > > wrote:
> > > >
> > > > > Hi Alejandro, all-
> > > > >
> > > > > There seems to be agreement on the broad stroke description of the
> > > components needed to achieve pluggable token authentication (I'm sure
> > I'll
> > > be corrected if that isn't the case). However, discussion of the
> details
> > of
> > > those components doesn't seem to be moving forward. I think this is
> > because
> > > the details are really best understood through code. I also see *a*
> (i.e.
> > > one of many possible) token format and pluggable authentication
> > mechanisms
> > > within the RPC layer as components that can have immediate benefit to
> > > Hadoop users AND still allow flexibility in the larger design. So, I
> > think
> > > the best way to move the conversation of "what we are aiming for"
> forward
> > > is to start looking at code for these components. I am especially
> > > interested in moving forward with pluggable authentication mechanisms
> > > within the RPC layer and would love to see what others have done in
> this
> > > area (if anything).
> > > > >
> > > > > Thanks.
> > > > >
> > > > > -Brian
> > > > >
> > > > > -----Original Message-----
> > > > > From: Alejandro Abdelnur [mailto:tucu@cloudera.com]
> > > > > Sent: Wednesday, July 10, 2013 8:15 AM
> > > > > To: Larry McCay
> > > > > Cc: common-dev@hadoop.apache.org; daryn@yahoo-inc.com; Kai Zheng
> > > > > Subject: Re: [DISCUSS] Hadoop SSO/Token Server Components
> > > > >
> > > > > Larry, all,
> > > > >
> > > > > Still is not clear to me what is the end state we are aiming for,
> or
> > > that we even agree on that.
> > > > >
> > > > > IMO, Instead trying to agree what to do, we should first  agree on
> > the
> > > final state, then we see what should be changed to there there, then we
> > see
> > > how we change things to get there.
> > > > >
> > > > > The different documents out there focus more on how.
> > > > >
> > > > > We not try to say how before we know what.
> > > > >
> > > > > Thx.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Jul 10, 2013 at 6:42 AM, Larry McCay <
> lmccay@hortonworks.com
> > >
> > > wrote:
> > > > >
> > > > >> All -
> > > > >>
> > > > >> After combing through this thread - as well as the summit session
> > > > >> summary thread, I think that we have the following two items that
> we
> > > > >> can probably move forward with:
> > > > >>
> > > > >> 1. TokenAuth method - assuming this means the pluggable
> > > > >> authentication mechanisms within the RPC layer (2 votes: Kai and
> > > > >> Kyle) 2. An actual Hadoop Token format (2 votes: Brian and myself)
> > > > >>
> > > > >> I propose that we attack both of these aspects as one. Let's
> provide
> > > > >> the structure and interfaces of the pluggable framework for use in
> > > > >> the RPC layer through leveraging Daryn's pluggability work and POC
> > it
> > > > >> with a particular token format (not necessarily the only format
> ever
> > > > >> supported - we just need one to start). If there has already been
> > > > >> work done in this area by anyone then please speak up and commit
> to
> > > > >> providing a patch - so that we don't duplicate effort.
> > > > >>
> > > > >> @Daryn - is there a particular Jira or set of Jiras that we can
> look
> > > > >> at to discern the pluggability mechanism details? Documentation of
> > it
> > > > >> would be great as well.
> > > > >> @Kai - do you have existing code for the pluggable token
> > > > >> authentication mechanism - if not, we can take a stab at
> > representing
> > > > >> it with interfaces and/or POC code.
> > > > >> I can standup and say that we have a token format that we have
> been
> > > > >> working with already and can provide a patch that represents it
> as a
> > > > >> contribution to test out the pluggable tokenAuth.
> > > > >>
> > > > >> These patches will provide progress toward code being the central
> > > > >> discussion vehicle. As a community, we can then incrementally
> build
> > > > >> on that foundation in order to collaboratively deliver the common
> > > vision.
> > > > >>
> > > > >> In the absence of any other home for posting such patches, let's
> > > > >> assume that they will be attached to HADOOP-9392 - or a dedicated
> > > > >> subtask for this particular aspect/s - I will leave that detail to
> > > Kai.
> > > > >>
> > > > >> @Alejandro, being the only voice on this thread that isn't
> > > > >> represented in the votes above, please feel free to agree or
> > disagree
> > > with this direction.
> > > > >>
> > > > >> thanks,
> > > > >>
> > > > >> --larry
> > > > >>
> > > > >> On Jul 5, 2013, at 3:24 PM, Larry McCay <lmccay@hortonworks.com>
> > > wrote:
> > > > >>
> > > > >>> Hi Andy -
> > > > >>>
> > > > >>>> Happy Fourth of July to you and yours.
> > > > >>>
> > > > >>> Same to you and yours. :-)
> > > > >>> We had some fun in the sun for a change - we've had nothing but
> > rain
> > > > >>> on
> > > > >> the east coast lately.
> > > > >>>
> > > > >>>> My concern here is there may have been a misinterpretation or
> lack
> > > > >>>> of consensus on what is meant by "clean slate"
> > > > >>>
> > > > >>>
> > > > >>> Apparently so.
> > > > >>> On the pre-summit call, I stated that I was interested in
> > > > >>> reconciling
> > > > >> the jiras so that we had one to work from.
> > > > >>>
> > > > >>> You recommended that we set them aside for the time being - with
> > the
> > > > >> understanding that work would continue on your side (and our's as
> > > > >> well) - and approach the community discussion from a clean slate.
> > > > >>> We seemed to do this at the summit session quite well.
> > > > >>> It was my understanding that this community discussion would live
> > > > >>> beyond
> > > > >> the summit and continue on this list.
> > > > >>>
> > > > >>> While closing the summit session we agreed to follow up on
> > > > >>> common-dev
> > > > >> with first a summary then a discussion of the moving parts.
> > > > >>>
> > > > >>> I never expected the previous work to be abandoned and fully
> > > > >>> expected it
> > > > >> to inform the discussion that happened here.
> > > > >>>
> > > > >>> If you would like to reframe what clean slate was supposed to
> mean
> > > > >>> or
> > > > >> describe what it means now - that would be welcome - before I
> waste
> > > > >> anymore time trying to facilitate a community discussion that is
> > > > >> apparently not wanted.
> > > > >>>
> > > > >>>> Nowhere in this
> > > > >>>> picture are self appointed "master JIRAs" and such, which have
> > been
> > > > >>>> disappointing to see crop up, we should be collaboratively
> coding
> > > > >>>> not planting flags.
> > > > >>>
> > > > >>> I don't know what you mean by self-appointed master JIRAs.
> > > > >>> It has certainly not been anyone's intention to disappoint.
> > > > >>> Any mention of a new JIRA was just to have a clear context to
> > gather
> > > > >>> the
> > > > >> agreed upon points - previous and/or existing JIRAs would easily
> be
> > > linked.
> > > > >>>
> > > > >>> Planting flags... I need to go back and read my discussion point
> > > > >>> about the
> > > > >> JIRA and see how this is the impression that was made.
> > > > >>> That is not how I define success. The only flags that count is
> > code.
> > > > >> What we are lacking is the roadmap on which to put the code.
> > > > >>>
> > > > >>>> I read Kai's latest document as something approaching today's
> > > > >>>> consensus
> > > > >> (or
> > > > >>>> at least a common point of view?) rather than a historical
> > document.
> > > > >>>> Perhaps he and it can be given equal share of the consideration.
> > > > >>>
> > > > >>> I definitely read it as something that has evolved into something
> > > > >> approaching what we have been talking about so far. There has not
> > > > >> however been enough discussion anywhere near the level of detail
> in
> > > > >> that document and more details are needed for each component in
> the
> > > design.
> > > > >>> Why the work in that document should not be fed into the
> community
> > > > >> discussion as anyone else's would be - I fail to understand.
> > > > >>>
> > > > >>> My suggestion continues to be that you should take that document
> > and
> > > > >> speak to the inventory of moving parts as we agreed.
> > > > >>> As these are agreed upon, we will ensure that the appropriate
> > > > >>> subtasks
> > > > >> are filed against whatever JIRA is to host them - don't really
> care
> > > > >> much which it is.
> > > > >>>
> > > > >>> I don't really want to continue with two separate JIRAs - as I
> > > > >>> stated
> > > > >> long ago - but until we understand what the pieces are and how
> they
> > > > >> relate then they can't be consolidated.
> > > > >>> Even if 9533 ended up being repurposed as the server instance of
> > the
> > > > >> work - it should be a subtask of a larger one - if that is to be
> > > > >> 9392, so be it.
> > > > >>> We still need to define all the pieces of the larger picture
> before
> > > > >>> that
> > > > >> can be done.
> > > > >>>
> > > > >>> What I thought was the clean slate approach to the discussion
> > seemed
> > > > >>> a
> > > > >> very reasonable way to make all this happen.
> > > > >>> If you would like to restate what you intended by it or something
> > > > >>> else
> > > > >> equally as reasonable as a way to move forward that would be
> > awesome.
> > > > >>>
> > > > >>> I will be happy to work toward the roadmap with everyone once it
> is
> > > > >> articulated, understood and actionable.
> > > > >>> In the meantime, I have work to do.
> > > > >>>
> > > > >>> thanks,
> > > > >>>
> > > > >>> --larry
> > > > >>>
> > > > >>> BTW - I meant to quote you in an earlier response and ended up
> > > > >>> saying it
> > > > >> was Aaron instead. Not sure what happened there. :-)
> > > > >>>
> > > > >>> On Jul 4, 2013, at 2:40 PM, Andrew Purtell <apurtell@apache.org>
> > > wrote:
> > > > >>>
> > > > >>>> Hi Larry (and all),
> > > > >>>>
> > > > >>>> Happy Fourth of July to you and yours.
> > > > >>>>
> > > > >>>> In our shop Kai and Tianyou are already doing the coding, so I'd
> > > > >>>> defer
> > > > >> to
> > > > >>>> them on the detailed points.
> > > > >>>>
> > > > >>>> My concern here is there may have been a misinterpretation or
> lack
> > > > >>>> of consensus on what is meant by "clean slate". Hopefully that
> can
> > > > >>>> be
> > > > >> quickly
> > > > >>>> cleared up. Certainly we did not mean ignore all that came
> before.
> > > > >>>> The
> > > > >> idea
> > > > >>>> was to reset discussions to find common ground and new direction
> > > > >>>> where
> > > > >> we
> > > > >>>> are working together, not in conflict, on an agreed upon set of
> > > > >>>> design points and tasks. There's been a lot of good discussion
> and
> > > > >>>> design preceeding that we should figure out how to port over.
> > > > >>>> Nowhere in this picture are self appointed "master JIRAs" and
> > such,
> > > > >>>> which have been disappointing to see crop up, we should be
> > > > >>>> collaboratively coding not planting flags.
> > > > >>>>
> > > > >>>> I read Kai's latest document as something approaching today's
> > > > >>>> consensus
> > > > >> (or
> > > > >>>> at least a common point of view?) rather than a historical
> > document.
> > > > >>>> Perhaps he and it can be given equal share of the consideration.
> > > > >>>>
> > > > >>>>
> > > > >>>> On Wednesday, July 3, 2013, Larry McCay wrote:
> > > > >>>>
> > > > >>>>> Hey Andrew -
> > > > >>>>>
> > > > >>>>> I largely agree with that statement.
> > > > >>>>> My intention was to let the differences be worked out within
> the
> > > > >>>>> individual components once they were identified and subtasks
> > > created.
> > > > >>>>>
> > > > >>>>> My reference to HSSO was really referring to a SSO *server*
> based
> > > > >> design
> > > > >>>>> which was not clearly articulated in the earlier documents.
> > > > >>>>> We aren't trying to compare and contrast one design over
> another
> > > > >> anymore.
> > > > >>>>>
> > > > >>>>> Let's move this collaboration along as we've mapped out and the
> > > > >>>>> differences in the details will reveal themselves and be
> > addressed
> > > > >> within
> > > > >>>>> their components.
> > > > >>>>>
> > > > >>>>> I've actually been looking forward to you weighing in on the
> > > > >>>>> actual discussion points in this thread.
> > > > >>>>> Could you do that?
> > > > >>>>>
> > > > >>>>> At this point, I am most interested in your thoughts on a
> single
> > > > >>>>> jira
> > > > >> to
> > > > >>>>> represent all of this work and whether we should start
> discussing
> > > > >>>>> the
> > > > >> SSO
> > > > >>>>> Tokens.
> > > > >>>>> If you think there are discussion points missing from that
> list,
> > > > >>>>> feel
> > > > >> free
> > > > >>>>> to add to it.
> > > > >>>>>
> > > > >>>>> thanks,
> > > > >>>>>
> > > > >>>>> --larry
> > > > >>>>>
> > > > >>>>> On Jul 3, 2013, at 7:35 PM, Andrew Purtell <
> apurtell@apache.org>
> > > > >> wrote:
> > > > >>>>>
> > > > >>>>>> Hi Larry,
> > > > >>>>>>
> > > > >>>>>> Of course I'll let Kai speak for himself. However, let me
> point
> > > > >>>>>> out
> > > > >> that,
> > > > >>>>>> while the differences between the competing JIRAs have been
> > > > >>>>>> reduced
> > > > >> for
> > > > >>>>>> sure, there were some key differences that didn't just
> > disappear.
> > > > >>>>>> Subsequent discussion will make that clear. I also disagree
> with
> > > > >>>>>> your characterization that we have simply endorsed all of the
> > > > >>>>>> design
> > > > >> decisions
> > > > >>>>>> of the so-called HSSO, this is taking a mile from an inch. We
> > are
> > > > >> here to
> > > > >>>>>> engage in a collaborative process as peers. I've been
> encouraged
> > > > >>>>>> by
> > > > >> the
> > > > >>>>>> spirit of the discussions up to this point and hope that can
> > > > >>>>>> continue beyond one design summit.
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> On Wed, Jul 3, 2013 at 1:10 PM, Larry McCay
> > > > >>>>>> <lmccay@hortonworks.com>
> > > > >>>>> wrote:
> > > > >>>>>>
> > > > >>>>>>> Hi Kai -
> > > > >>>>>>>
> > > > >>>>>>> I think that I need to clarify something...
> > > > >>>>>>>
> > > > >>>>>>> This is not an update for 9533 but a continuation of the
> > > > >>>>>>> discussions
> > > > >>>>> that
> > > > >>>>>>> are focused on a fresh look at a SSO for Hadoop.
> > > > >>>>>>> We've agreed to leave our previous designs behind and
> therefore
> > > > >>>>>>> we
> > > > >>>>> aren't
> > > > >>>>>>> really seeing it as an HSSO layered on top of TAS approach or
> > an
> > > > >> HSSO vs
> > > > >>>>>>> TAS discussion.
> > > > >>>>>>>
> > > > >>>>>>> Your latest design revision actually makes it clear that you
> > are
> > > > >>>>>>> now targeting exactly what was described as HSSO - so
> comparing
> > > > >>>>>>> and
> > > > >>>>> contrasting
> > > > >>>>>>> is not going to add any value.
> > > > >>>>>>>
> > > > >>>>>>> What we need you to do at this point, is to look at those
> > > > >>>>>>> high-level components described on this thread and comment on
> > > > >>>>>>> whether we need additional components or any that are listed
> > > > >>>>>>> that don't seem
> > > > >> necessary
> > > > >>>>> to
> > > > >>>>>>> you and why.
> > > > >>>>>>> In other words, we need to define and agree on the work that
> > has
> > > > >>>>>>> to
> > > > >> be
> > > > >>>>>>> done.
> > > > >>>>>>>
> > > > >>>>>>> We also need to determine those components that need to be
> done
> > > > >> before
> > > > >>>>>>> anything else can be started.
> > > > >>>>>>> I happen to agree with Brian that #4 Hadoop SSO Tokens are
> > > > >>>>>>> central to
> > > > >>>>> all
> > > > >>>>>>> the other components and should probably be defined and POC'd
> > in
> > > > >> short
> > > > >>>>>>> order.
> > > > >>>>>>>
> > > > >>>>>>> Personally, I think that continuing the separation of 9533
> and
> > > > >>>>>>> 9392
> > > > >> will
> > > > >>>>>>> do this effort a disservice. There doesn't seem to be enough
> > > > >> differences
> > > > >>>>>>> between the two to justify separate jiras anymore. It may be
> > > > >>>>>>> best to
> > > > >>>>> file a
> > > > >>>>>>> new one that reflects a single vision without the extra cruft
> > > > >>>>>>> that
> > > > >> has
> > > > >>>>>>> built up in either of the existing ones. We would certainly
> > > > >>>>>>> reference
> > > > >>>>> the
> > > > >>>>>>> existing ones within the new one. This approach would align
> > with
> > > > >>>>>>> the
> > > > >>>>> spirit
> > > > >>>>>>> of the discussions up to this point.
> > > > >>>>>>>
> > > > >>>>>>> I am prepared to start a discussion around the shape of the
> two
> > > > >> Hadoop
> > > > >>>>> SSO
> > > > >>>>>>> tokens: identity and access. If this is what others feel the
> > > > >>>>>>> next
> > > > >> topic
> > > > >>>>>>> should be.
> > > > >>>>>>> If we can identify a jira home for it, we can do it there -
> > > > >> otherwise we
> > > > >>>>>>> can create another DISCUSS thread for it.
> > > > >>>>>>>
> > > > >>>>>>> thanks,
> > > > >>>>>>>
> > > > >>>>>>> --larry
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>> On Jul 3, 2013, at 2:39 PM, "Zheng, Kai" <
> kai.zheng@intel.com>
> > > > >> wrote:
> > > > >>>>>>>
> > > > >>>>>>>> Hi Larry,
> > > > >>>>>>>>
> > > > >>>>>>>> Thanks for the update. Good to see that with this update we
> > are
> > > > >>>>>>>> now
> > > > >>>>>>> aligned on most points.
> > > > >>>>>>>>
> > > > >>>>>>>> I have also updated our TokenAuth design in HADOOP-9392. The
> > > > >>>>>>>> new
> > > > >>>>>>> revision incorporates feedback and suggestions in related
> > > > >>>>>>> discussion
> > > > >>>>> with
> > > > >>>>>>> the community, particularly from Microsoft and others
> attending
> > > > >>>>>>> the Security design lounge session at the Hadoop summit.
> > Summary
> > > > >>>>>>> of the
> > > > >>>>> changes:
> > > > >>>>>>>> 1.    Revised the approach to now use two tokens, Identity
> > Token
> > > > >> plus
> > > > >>>>>>> Access Token, particularly considering our authorization
> > > > >>>>>>> framework
> > > > >> and
> > > > >>>>>>> compatibility with HSSO;
> > > > >>>>>>>> 2.    Introduced Authorization Server (AS) from our
> > > authorization
> > > > >>>>>>> framework into the flow that issues access tokens for clients
> > > > >>>>>>> with
> > > > >>>>> identity
> > > > >>>>>>> tokens to access services;
> > > > >>>>>>>> 3.    Refined proxy access token and the proxy/impersonation
> > > flow;
> > > > >>>>>>>> 4.    Refined the browser web SSO flow regarding access to
> > > Hadoop
> > > > >> web
> > > > >>>>>>> services;
> > > > >>>>>>>> 5.    Added Hadoop RPC access flow regard
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> --
> > > > >>>> Best regards,
> > > > >>>>
> > > > >>>> - Andy
> > > > >>>>
> > > > >>>> Problems worthy of attack prove their worth by hitting back. -
> > Piet
> > > > >>>> Hein (via Tom White)
> > > > >>>
> > > > >>
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Alejandro
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > <Iteration1PluggableUserAuthenticationandFederation.pdf>
> > >
> > >
> >
> >
> > --
> > Alejandro
> >
>



-- 
Alejandro

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message