Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
MIME-Version: 1.0
Reply-To: varunsaxena@apache.org
In-Reply-To: <CAOScs9aWaTeTv2cnibf8nHzyx-io_Yd2KNZk9s2QxRRgsrFLLg@mail.gmail.com>
References: <CAE=b_fbLT2J+Ezb4wqdN_UwBiG1Sd5kpqGaw+9Br__zou5yNTQ@mail.gmail.com>
 <CACYiTuhumHKVSb1HZqfGzcWsVjjPBHA0wSUPBONrc4BH+bKeDw@mail.gmail.com>
 <CAP4BK9QiZqCZ_8kXOpX6Mbn+Az5W-M+MPk-q0J_33946TjBjWA@mail.gmail.com>
 <CADbBEns4+oXrmpuZFv3aFdVUV9UdW7To6fzrjY96OFLPwjfcwQ@mail.gmail.com> <CAOScs9aWaTeTv2cnibf8nHzyx-io_Yd2KNZk9s2QxRRgsrFLLg@mail.gmail.com>
From: "varunsaxena@apache.org" <varun.saxena.apache@gmail.com>
Date: Sat, 26 Aug 2017 02:58:56 +0530
Message-ID: <CANyB6pu4ELzi70VzMzna1VGt_OaKntWXM+pn+Q_aHUjwD6NiJg@mail.gmail.com>
Subject: Re: [VOTE] Merge feature branch YARN-5355 (Timeline Service v2) to trunk
To: Subramaniam V K <subru.vk@gmail.com>
Cc: Sangjin Lee <sjlee@apache.org>, Haibo Chen <haibochen@cloudera.com>,
	Sunil G <sunilg@apache.org>, Vrushali Channapattan <vrushalic2016@gmail.com>,
	"common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>,
	"yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>,
	"mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>,
	"hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>
Content-Type: multipart/alternative; boundary="94eb2c1161c0f6042e05579aa1a6"
archived-at: Fri, 25 Aug 2017 21:29:12 -0000

--94eb2c1161c0f6042e05579aa1a6
Content-Type: text/plain; charset="UTF-8"

Thanks Subru for voting.


> What are the timelines you are looking for getting this into branch-2?

We haven't yet decided on it and were thinking of discussing this in detail
within the team after merge to trunk.
The timelines would depend on whether we release whatever we merge to trunk
in 2.9 or would we want to get in few other features which people would
like to see in 2.9
This would require some discussion with the stakeholders.
We were thinking of having a short discussion with you guys as well to find
out whether there are any further gaps in ATSv2 with respect to federation
support and if they can be filled before 2.9 release.

Assuming 2.9 is targetted for October end, we would have to start a merge
vote in September end or October 1st week which leaves us with very little
time to take up large changes anyways.

We do maintain a branch-2 version of ATSv2(YARN-5355_branch2) though which
we rebase with branch-2 regularly. So, if we decide to merge in branch-2
without any additional changes, we would be able to go for branch-2 merge
discussion and vote almost immediately.

Regards,
Varun Saxena.


On Sat, Aug 26, 2017 at 2:00 AM, Subramaniam V K <subru.vk@gmail.com> wrote:

> +1 (binding).
>
> I have been following the effort and had few design discussions around the
> team especially about how it integrates with Federation. Overall I feel
> it's a welcome improvement to YARN.
>
> What are the timelines you are looking for getting this into branch-2?
>
> Thanks,
> Subru
>
> On Fri, Aug 25, 2017 at 10:04 AM, Sangjin Lee <sjlee@apache.org> wrote:
>
> > +1 (binding)
> >
> > I've built the current branch, and checked out a few basic areas
> including
> > documentation. Also perused the most recent changes that went in.
> >
> > Thanks much for the great team work! I look forward to seeing it in
> action.
> >
> > Regards,
> > Sangjin
> >
> > On Fri, Aug 25, 2017 at 9:27 AM, Haibo Chen <haibochen@cloudera.com>
> > wrote:
> >
> > > +1 from my side.
> > >
> > > More from the perspective of ensuring there is no impact of ATSv2 when
> it
> > > is off (by default), I deployed the latest YARN-5355 bits into a few
> > > clusters and ran internal Smoke tests. The tests shows no impact when
> > ATSv2
> > > is off.
> > >
> > > Best,
> > > Haibo
> > >
> > > On Thu, Aug 24, 2017 at 7:51 AM, Sunil G <sunilg@apache.org> wrote:
> > >
> > > > Thank you very much Vrushali, Rohith, Varun and other folks who made
> > this
> > > > happen. Great work, really appreciate the same!!
> > > >
> > > > +1 (binding) from my side:
> > > >
> > > > # Tested ATSv2 cluster in a secure cluster. Ran some basic jobs
> > > > # Accessed new YARN UI which shows various flows/flow activity etc.
> > Seems
> > > > fine.
> > > > # Based on code, looks like all apis are compatible.
> > > > # REST api docs looks fine as well, I guess we could improve that a
> bit
> > > > more post merge as well.
> > > > # Adding to additional thoughts which are discussed here, native
> > service
> > > > also could publish events to atsv2. I think that work is also
> happened
> > in
> > > > branch.
> > > >
> > > > Looking forward to a much wider adoption of ATSv2 with more projects.
> > > >
> > > > Thanks
> > > > Sunil
> > > >
> > > >
> > > > On Tue, Aug 22, 2017 at 12:02 PM Vrushali Channapattan <
> > > > vrushalic2016@gmail.com> wrote:
> > > >
> > > > > Hi folks,
> > > > >
> > > > > Per earlier discussion [1], I'd like to start a formal vote to
> merge
> > > > > feature branch YARN-5355 [2] (Timeline Service v.2) to trunk. The
> > vote
> > > > will
> > > > > run for 7 days, and will end August 29 11:00 PM PDT.
> > > > >
> > > > > We have previously completed one merge onto trunk [3] and Timeline
> > > > Service
> > > > > v2 has been part of Hadoop release 3.0.0-alpha1.
> > > > >
> > > > > Since then, we have been working on extending the capabilities of
> > > > Timeline
> > > > > Service v2 in a feature branch [2] for a while, and we are
> reasonably
> > > > > confident that the state of the feature meets the criteria to be
> > merged
> > > > > onto trunk and we'd love folks to get their hands on it in a test
> > > > capacity
> > > > > and provide valuable feedback so that we can make it
> > production-ready.
> > > > >
> > > > > In a nutshell, Timeline Service v.2 delivers significant
> scalability
> > > and
> > > > > usability improvements based on a new architecture. What we would
> > like
> > > to
> > > > > merge to trunk is termed "alpha 2" (milestone 2). The feature has a
> > > > > complete end-to-end read/write flow with security and read level
> > > > > authorization via whitelists. You should be able to start setting
> it
> > up
> > > > and
> > > > > testing it.
> > > > >
> > > > > At a high level, the following are the key features that have been
> > > > > implemented since alpha1:
> > > > > - Security via Kerberos Authentication and delegation tokens
> > > > > - Read side simple authorization via whitelist
> > > > > - Client configurable entity sort ordering
> > > > > - Richer REST APIs for apps, app attempts, containers, fetching
> > metrics
> > > > by
> > > > > timerange, pagination, sub-app entities
> > > > > - Support for storing sub-application entities (entities that exist
> > > > outside
> > > > > the scope of an application)
> > > > > - Configurable TTLs (time-to-live) for tables, configurable table
> > > > prefixes,
> > > > > configurable hbase cluster
> > > > > - Flow level aggregations done as dynamic (table level)
> coprocessors
> > > > > - Uses latest stable HBase release 1.2.6
> > > > >
> > > > > There are a total of 82 subtasks that were completed as part of
> this
> > > > > effort.
> > > > >
> > > > > We paid close attention to ensure that once disabled Timeline
> Service
> > > v.2
> > > > > does not impact existing functionality when disabled (by default).
> > > > >
> > > > > Special thanks to a team of folks who worked hard and contributed
> > > towards
> > > > > this effort with patches, reviews and guidance: Rohith Sharma K S,
> > > Varun
> > > > > Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar Vavilapalli,
> Joep
> > > > > Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal Stack.
> > > > >
> > > > > Regards,
> > > > > Vrushali
> > > > >
> > > > > [1] http://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg27
> > > 383.html
> > > > > [2] https://issues.apache.org/jira/browse/YARN-5355
> > > > > [3] https://issues.apache.org/jira/browse/YARN-2928
> > > > > [4] https://github.com/apache/hadoop/commits/YARN-5355
> > > > >
> > > >
> > >
> >
>

--94eb2c1161c0f6042e05579aa1a6--