Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of tucu@cloudera.com designates
 209.85.128.49 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CACBYxKK_yLRf16H-sg1+CnbPpEPxnMJZxzLAeKFfYD8t1J0aVg@mail.gmail.com>
References: 
 <CAJS6+b6zRnpZky_gSc5R-fbbjmpu4KsyM=jot4xS=9Oz_g-HPg@mail.gmail.com>
 <CA+01ahiwVHYVmDRSLcOYnqt3w+xGq_p_N41JgwYk4GD6w7zG3A@mail.gmail.com>
 <CACBYxKK_yLRf16H-sg1+CnbPpEPxnMJZxzLAeKFfYD8t1J0aVg@mail.gmail.com>
From: Alejandro Abdelnur <tucu@cloudera.com>
Date: Mon, 6 Jan 2014 15:02:57 -0800
Message-ID: 
 <CAJs-t7OQLZyfZ3_i2GYEsoXAHi22=9fWSA_RsbQ-BC-iWVQ4pQ@mail.gmail.com>
Subject: Re: [Discussion] Merge YARN-321 into Branch-2
To: "yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>
Cc: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=001a1133ba2c88f48104ef554447

--001a1133ba2c88f48104ef554447
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

This is great news. A few things to consider before doing the merge:

1* The merge must be done in trunk first (then, ideally, from trunk into
branch-2)
2* The last update of YARN-321 was done in NOV10, this was done from
branch-2 (that seems a NIT as it should be against trunk)
3* IMO, until we don=92t have security, we should merge into trunk only

I would like to see #1 and #2 taken care before making a decision. The
reason for this is that if the source changes outside of the AHS are too
pervasive, then we may end up be making difficult backports from trunk to
branch-2 and release branches because of the delta.

Can we get a rebase of the YARN-321 to the head of trunk to see if my
concerns are valid or not?

Thanks.

Alejandro


On Mon, Jan 6, 2014 at 1:11 AM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:

> Very excited for this feature and appreciative of all the work put into
> this.  Reviewed the JIRA and my only two remaining concerns are about
> documentation and API stability.
>
> Regarding doc, while we don't necessarily need full documentation before
> merging, my feeling is that we should at least have a page on it that wil=
l
> allow give cluster operators a sketch of its role, APIs, and implications=
.
>  If this doesn't exist yet, would be happy to help with reviewing.
>
> Regarding APIs, I imagine some will jump on this as soon as it becomes pa=
rt
> of a release as it's a fairly essential feature for non-MR apps.  My
> opinion is that we should mark each API @Stable unless there is a strong
> reason for it not to be.  When I say APIs, I am thinking of the REST APIs=
,
> RPC interface, and RM<->AHS shared-bus.
>
> -Sandy
>
>
> On Sun, Jan 5, 2014 at 8:33 PM, lohit <lohit.vijayarenu@gmail.com> wrote:
>
> > +1 to merge into branch2 now.
> > On Jan 5, 2014 6:22 PM, "Zhijie Shen" <zshen@hortonworks.com> wrote:
> >
> > > Hi folks,
> > >
> > > Majority of the functionality of Application History Server has been
> > > completed on branch YARN-321. AHS can now work end-to-end.
> > ResourceManager
> > > records the historical information of the application, the applicatio=
n
> > > attempt and the container in terms of events via a history writer on =
a
> > > separate thread. The historical information is going to be persisted =
on
> > > HDFS. On the other side, an application history server runs as a
> separate
> > > process, and it allows users to access the historical information via
> RPC
> > > interface, web UI and REST APIs.
> > >
> > > According to the proposal, the only major missing piece is security.
> > Except
> > > it, YARN-321 should be pretty much ready to be merged to Branch-2. We
> > > propose to merge YARN-321 to Branch-2 now, such that we can prevent
> > > YARN-321 from falling behind too much, and reduce the effort of editi=
ng
> > > merge conflicts. After merge, we can continue on security, refactorin=
g
> > > duplicate code, fixing bugs and other improvements.
> > >
> > > Anyone who is interested in looking at AHS can review/play with
> YARN-321
> > > branch: http://svn.apache.org/viewvc/hadoop/common/branches/YARN-321/
> > >
> > > You can also have a look at the latest design doc:
> > >
> > >
> >
> https://issues.apache.org/jira/secure/attachment/12619638/Generic%20Appli=
cation%20History%20-%20Design-20131219.pdf
> > >
> > > If there are no objections, we'll push towards updating the branch,
> > running
> > > the patch through Jenkins and getting ready for merge vote by the end
> of
> > > this week.
> > >
> > > Thanks,
> > > Zhijie
> > >
> > > --
> > > Zhijie Shen
> > > Hortonworks Inc.
> > > http://hortonworks.com/
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidentia=
l,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notifie=
d
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
>


--=20
Alejandro

--001a1133ba2c88f48104ef554447--