hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alejandro Abdelnur <t...@cloudera.com>
Subject Re: [DISCUSS] Hadoop SSO/Token Server Components
Date Wed, 10 Jul 2013 15:14:40 GMT
Larry, all,

Still is not clear to me what is the end state we are aiming for, or that
we even agree on that.

IMO, Instead trying to agree what to do, we should first  agree on the
final state, then we see what should be changed to there there, then we see
how we change things to get there.

The different documents out there focus more on how.

We not try to say how before we know what.

Thx.




On Wed, Jul 10, 2013 at 6:42 AM, Larry McCay <lmccay@hortonworks.com> wrote:

> All -
>
> After combing through this thread - as well as the summit session summary
> thread, I think that we have the following two items that we can probably
> move forward with:
>
> 1. TokenAuth method - assuming this means the pluggable authentication
> mechanisms within the RPC layer (2 votes: Kai and Kyle)
> 2. An actual Hadoop Token format (2 votes: Brian and myself)
>
> I propose that we attack both of these aspects as one. Let's provide the
> structure and interfaces of the pluggable framework for use in the RPC
> layer through leveraging Daryn's pluggability work and POC it with a
> particular token format (not necessarily the only format ever supported -
> we just need one to start). If there has already been work done in this
> area by anyone then please speak up and commit to providing a patch - so
> that we don't duplicate effort.
>
> @Daryn - is there a particular Jira or set of Jiras that we can look at to
> discern the pluggability mechanism details? Documentation of it would be
> great as well.
> @Kai - do you have existing code for the pluggable token authentication
> mechanism - if not, we can take a stab at representing it with interfaces
> and/or POC code.
> I can standup and say that we have a token format that we have been
> working with already and can provide a patch that represents it as a
> contribution to test out the pluggable tokenAuth.
>
> These patches will provide progress toward code being the central
> discussion vehicle. As a community, we can then incrementally build on that
> foundation in order to collaboratively deliver the common vision.
>
> In the absence of any other home for posting such patches, let's assume
> that they will be attached to HADOOP-9392 - or a dedicated subtask for this
> particular aspect/s - I will leave that detail to Kai.
>
> @Alejandro, being the only voice on this thread that isn't represented in
> the votes above, please feel free to agree or disagree with this direction.
>
> thanks,
>
> --larry
>
> On Jul 5, 2013, at 3:24 PM, Larry McCay <lmccay@hortonworks.com> wrote:
>
> > Hi Andy -
> >
> >> Happy Fourth of July to you and yours.
> >
> > Same to you and yours. :-)
> > We had some fun in the sun for a change - we've had nothing but rain on
> the east coast lately.
> >
> >> My concern here is there may have been a misinterpretation or lack of
> >> consensus on what is meant by "clean slate"
> >
> >
> > Apparently so.
> > On the pre-summit call, I stated that I was interested in reconciling
> the jiras so that we had one to work from.
> >
> > You recommended that we set them aside for the time being - with the
> understanding that work would continue on your side (and our's as well) -
> and approach the community discussion from a clean slate.
> > We seemed to do this at the summit session quite well.
> > It was my understanding that this community discussion would live beyond
> the summit and continue on this list.
> >
> > While closing the summit session we agreed to follow up on common-dev
> with first a summary then a discussion of the moving parts.
> >
> > I never expected the previous work to be abandoned and fully expected it
> to inform the discussion that happened here.
> >
> > If you would like to reframe what clean slate was supposed to mean or
> describe what it means now - that would be welcome - before I waste anymore
> time trying to facilitate a community discussion that is apparently not
> wanted.
> >
> >> Nowhere in this
> >> picture are self appointed "master JIRAs" and such, which have been
> >> disappointing to see crop up, we should be collaboratively coding not
> >> planting flags.
> >
> > I don't know what you mean by self-appointed master JIRAs.
> > It has certainly not been anyone's intention to disappoint.
> > Any mention of a new JIRA was just to have a clear context to gather the
> agreed upon points - previous and/or existing JIRAs would easily be linked.
> >
> > Planting flagsā€¦ I need to go back and read my discussion point about the
> JIRA and see how this is the impression that was made.
> > That is not how I define success. The only flags that count is code.
> What we are lacking is the roadmap on which to put the code.
> >
> >> I read Kai's latest document as something approaching today's consensus
> (or
> >> at least a common point of view?) rather than a historical document.
> >> Perhaps he and it can be given equal share of the consideration.
> >
> > I definitely read it as something that has evolved into something
> approaching what we have been talking about so far. There has not however
> been enough discussion anywhere near the level of detail in that document
> and more details are needed for each component in the design.
> > Why the work in that document should not be fed into the community
> discussion as anyone else's would be - I fail to understand.
> >
> > My suggestion continues to be that you should take that document and
> speak to the inventory of moving parts as we agreed.
> > As these are agreed upon, we will ensure that the appropriate subtasks
> are filed against whatever JIRA is to host them - don't really care much
> which it is.
> >
> > I don't really want to continue with two separate JIRAs - as I stated
> long ago - but until we understand what the pieces are and how they relate
> then they can't be consolidated.
> > Even if 9533 ended up being repurposed as the server instance of the
> work - it should be a subtask of a larger one - if that is to be 9392, so
> be it.
> > We still need to define all the pieces of the larger picture before that
> can be done.
> >
> > What I thought was the clean slate approach to the discussion seemed a
> very reasonable way to make all this happen.
> > If you would like to restate what you intended by it or something else
> equally as reasonable as a way to move forward that would be awesome.
> >
> > I will be happy to work toward the roadmap with everyone once it is
> articulated, understood and actionable.
> > In the meantime, I have work to do.
> >
> > thanks,
> >
> > --larry
> >
> > BTW - I meant to quote you in an earlier response and ended up saying it
> was Aaron instead. Not sure what happened there. :-)
> >
> > On Jul 4, 2013, at 2:40 PM, Andrew Purtell <apurtell@apache.org> wrote:
> >
> >> Hi Larry (and all),
> >>
> >> Happy Fourth of July to you and yours.
> >>
> >> In our shop Kai and Tianyou are already doing the coding, so I'd defer
> to
> >> them on the detailed points.
> >>
> >> My concern here is there may have been a misinterpretation or lack of
> >> consensus on what is meant by "clean slate". Hopefully that can be
> quickly
> >> cleared up. Certainly we did not mean ignore all that came before. The
> idea
> >> was to reset discussions to find common ground and new direction where
> we
> >> are working together, not in conflict, on an agreed upon set of design
> >> points and tasks. There's been a lot of good discussion and design
> >> preceeding that we should figure out how to port over. Nowhere in this
> >> picture are self appointed "master JIRAs" and such, which have been
> >> disappointing to see crop up, we should be collaboratively coding not
> >> planting flags.
> >>
> >> I read Kai's latest document as something approaching today's consensus
> (or
> >> at least a common point of view?) rather than a historical document.
> >> Perhaps he and it can be given equal share of the consideration.
> >>
> >>
> >> On Wednesday, July 3, 2013, Larry McCay wrote:
> >>
> >>> Hey Andrew -
> >>>
> >>> I largely agree with that statement.
> >>> My intention was to let the differences be worked out within the
> >>> individual components once they were identified and subtasks created.
> >>>
> >>> My reference to HSSO was really referring to a SSO *server* based
> design
> >>> which was not clearly articulated in the earlier documents.
> >>> We aren't trying to compare and contrast one design over another
> anymore.
> >>>
> >>> Let's move this collaboration along as we've mapped out and the
> >>> differences in the details will reveal themselves and be addressed
> within
> >>> their components.
> >>>
> >>> I've actually been looking forward to you weighing in on the actual
> >>> discussion points in this thread.
> >>> Could you do that?
> >>>
> >>> At this point, I am most interested in your thoughts on a single jira
> to
> >>> represent all of this work and whether we should start discussing the
> SSO
> >>> Tokens.
> >>> If you think there are discussion points missing from that list, feel
> free
> >>> to add to it.
> >>>
> >>> thanks,
> >>>
> >>> --larry
> >>>
> >>> On Jul 3, 2013, at 7:35 PM, Andrew Purtell <apurtell@apache.org>
> wrote:
> >>>
> >>>> Hi Larry,
> >>>>
> >>>> Of course I'll let Kai speak for himself. However, let me point out
> that,
> >>>> while the differences between the competing JIRAs have been reduced
> for
> >>>> sure, there were some key differences that didn't just disappear.
> >>>> Subsequent discussion will make that clear. I also disagree with your
> >>>> characterization that we have simply endorsed all of the design
> decisions
> >>>> of the so-called HSSO, this is taking a mile from an inch. We are
> here to
> >>>> engage in a collaborative process as peers. I've been encouraged by
> the
> >>>> spirit of the discussions up to this point and hope that can continue
> >>>> beyond one design summit.
> >>>>
> >>>>
> >>>>
> >>>> On Wed, Jul 3, 2013 at 1:10 PM, Larry McCay <lmccay@hortonworks.com>
> >>> wrote:
> >>>>
> >>>>> Hi Kai -
> >>>>>
> >>>>> I think that I need to clarify somethingā€¦
> >>>>>
> >>>>> This is not an update for 9533 but a continuation of the discussions
> >>> that
> >>>>> are focused on a fresh look at a SSO for Hadoop.
> >>>>> We've agreed to leave our previous designs behind and therefore
we
> >>> aren't
> >>>>> really seeing it as an HSSO layered on top of TAS approach or an
> HSSO vs
> >>>>> TAS discussion.
> >>>>>
> >>>>> Your latest design revision actually makes it clear that you are
now
> >>>>> targeting exactly what was described as HSSO - so comparing and
> >>> contrasting
> >>>>> is not going to add any value.
> >>>>>
> >>>>> What we need you to do at this point, is to look at those high-level
> >>>>> components described on this thread and comment on whether we need
> >>>>> additional components or any that are listed that don't seem
> necessary
> >>> to
> >>>>> you and why.
> >>>>> In other words, we need to define and agree on the work that has
to
> be
> >>>>> done.
> >>>>>
> >>>>> We also need to determine those components that need to be done
> before
> >>>>> anything else can be started.
> >>>>> I happen to agree with Brian that #4 Hadoop SSO Tokens are central
to
> >>> all
> >>>>> the other components and should probably be defined and POC'd in
> short
> >>>>> order.
> >>>>>
> >>>>> Personally, I think that continuing the separation of 9533 and 9392
> will
> >>>>> do this effort a disservice. There doesn't seem to be enough
> differences
> >>>>> between the two to justify separate jiras anymore. It may be best
to
> >>> file a
> >>>>> new one that reflects a single vision without the extra cruft that
> has
> >>>>> built up in either of the existing ones. We would certainly reference
> >>> the
> >>>>> existing ones within the new one. This approach would align with
the
> >>> spirit
> >>>>> of the discussions up to this point.
> >>>>>
> >>>>> I am prepared to start a discussion around the shape of the two
> Hadoop
> >>> SSO
> >>>>> tokens: identity and access. If this is what others feel the next
> topic
> >>>>> should be.
> >>>>> If we can identify a jira home for it, we can do it there -
> otherwise we
> >>>>> can create another DISCUSS thread for it.
> >>>>>
> >>>>> thanks,
> >>>>>
> >>>>> --larry
> >>>>>
> >>>>>
> >>>>> On Jul 3, 2013, at 2:39 PM, "Zheng, Kai" <kai.zheng@intel.com>
> wrote:
> >>>>>
> >>>>>> Hi Larry,
> >>>>>>
> >>>>>> Thanks for the update. Good to see that with this update we
are now
> >>>>> aligned on most points.
> >>>>>>
> >>>>>> I have also updated our TokenAuth design in HADOOP-9392. The
new
> >>>>> revision incorporates feedback and suggestions in related discussion
> >>> with
> >>>>> the community, particularly from Microsoft and others attending
the
> >>>>> Security design lounge session at the Hadoop summit. Summary of
the
> >>> changes:
> >>>>>> 1.    Revised the approach to now use two tokens, Identity Token
> plus
> >>>>> Access Token, particularly considering our authorization framework
> and
> >>>>> compatibility with HSSO;
> >>>>>> 2.    Introduced Authorization Server (AS) from our authorization
> >>>>> framework into the flow that issues access tokens for clients with
> >>> identity
> >>>>> tokens to access services;
> >>>>>> 3.    Refined proxy access token and the proxy/impersonation
flow;
> >>>>>> 4.    Refined the browser web SSO flow regarding access to Hadoop
> web
> >>>>> services;
> >>>>>> 5.    Added Hadoop RPC access flow regard
> >>
> >>
> >>
> >> --
> >> Best regards,
> >>
> >>  - Andy
> >>
> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> >> (via Tom White)
> >
>
>


-- 
Alejandro

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message