hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alejandro Abdelnur <t...@cloudera.com>
Subject Re: [Discussion] Merge YARN-321 into Branch-2
Date Mon, 06 Jan 2014 23:02:57 GMT
This is great news. A few things to consider before doing the merge:

1* The merge must be done in trunk first (then, ideally, from trunk into
branch-2)
2* The last update of YARN-321 was done in NOV10, this was done from
branch-2 (that seems a NIT as it should be against trunk)
3* IMO, until we don’t have security, we should merge into trunk only

I would like to see #1 and #2 taken care before making a decision. The
reason for this is that if the source changes outside of the AHS are too
pervasive, then we may end up be making difficult backports from trunk to
branch-2 and release branches because of the delta.

Can we get a rebase of the YARN-321 to the head of trunk to see if my
concerns are valid or not?

Thanks.

Alejandro



On Mon, Jan 6, 2014 at 1:11 AM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:

> Very excited for this feature and appreciative of all the work put into
> this.  Reviewed the JIRA and my only two remaining concerns are about
> documentation and API stability.
>
> Regarding doc, while we don't necessarily need full documentation before
> merging, my feeling is that we should at least have a page on it that will
> allow give cluster operators a sketch of its role, APIs, and implications.
>  If this doesn't exist yet, would be happy to help with reviewing.
>
> Regarding APIs, I imagine some will jump on this as soon as it becomes part
> of a release as it's a fairly essential feature for non-MR apps.  My
> opinion is that we should mark each API @Stable unless there is a strong
> reason for it not to be.  When I say APIs, I am thinking of the REST APIs,
> RPC interface, and RM<->AHS shared-bus.
>
> -Sandy
>
>
> On Sun, Jan 5, 2014 at 8:33 PM, lohit <lohit.vijayarenu@gmail.com> wrote:
>
> > +1 to merge into branch2 now.
> > On Jan 5, 2014 6:22 PM, "Zhijie Shen" <zshen@hortonworks.com> wrote:
> >
> > > Hi folks,
> > >
> > > Majority of the functionality of Application History Server has been
> > > completed on branch YARN-321. AHS can now work end-to-end.
> > ResourceManager
> > > records the historical information of the application, the application
> > > attempt and the container in terms of events via a history writer on a
> > > separate thread. The historical information is going to be persisted on
> > > HDFS. On the other side, an application history server runs as a
> separate
> > > process, and it allows users to access the historical information via
> RPC
> > > interface, web UI and REST APIs.
> > >
> > > According to the proposal, the only major missing piece is security.
> > Except
> > > it, YARN-321 should be pretty much ready to be merged to Branch-2. We
> > > propose to merge YARN-321 to Branch-2 now, such that we can prevent
> > > YARN-321 from falling behind too much, and reduce the effort of editing
> > > merge conflicts. After merge, we can continue on security, refactoring
> > > duplicate code, fixing bugs and other improvements.
> > >
> > > Anyone who is interested in looking at AHS can review/play with
> YARN-321
> > > branch: http://svn.apache.org/viewvc/hadoop/common/branches/YARN-321/
> > >
> > > You can also have a look at the latest design doc:
> > >
> > >
> >
> https://issues.apache.org/jira/secure/attachment/12619638/Generic%20Application%20History%20-%20Design-20131219.pdf
> > >
> > > If there are no objections, we'll push towards updating the branch,
> > running
> > > the patch through Jenkins and getting ready for merge vote by the end
> of
> > > this week.
> > >
> > > Thanks,
> > > Zhijie
> > >
> > > --
> > > Zhijie Shen
> > > Hortonworks Inc.
> > > http://hortonworks.com/
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
>



-- 
Alejandro

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message