Mailing-List: contact yarn-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
MIME-Version: 1.0
In-Reply-To: <CADbBEns4+oXrmpuZFv3aFdVUV9UdW7To6fzrjY96OFLPwjfcwQ@mail.gmail.com>
References: <CAE=b_fbLT2J+Ezb4wqdN_UwBiG1Sd5kpqGaw+9Br__zou5yNTQ@mail.gmail.com>
 <CACYiTuhumHKVSb1HZqfGzcWsVjjPBHA0wSUPBONrc4BH+bKeDw@mail.gmail.com>
 <CAP4BK9QiZqCZ_8kXOpX6Mbn+Az5W-M+MPk-q0J_33946TjBjWA@mail.gmail.com> <CADbBEns4+oXrmpuZFv3aFdVUV9UdW7To6fzrjY96OFLPwjfcwQ@mail.gmail.com>
From: Subramaniam V K <subru.vk@gmail.com>
Date: Fri, 25 Aug 2017 13:30:48 -0700
Message-ID: <CAOScs9aWaTeTv2cnibf8nHzyx-io_Yd2KNZk9s2QxRRgsrFLLg@mail.gmail.com>
Subject: Re: [VOTE] Merge feature branch YARN-5355 (Timeline Service v2) to trunk
To: Sangjin Lee <sjlee@apache.org>
Cc: Haibo Chen <haibochen@cloudera.com>, Sunil G <sunilg@apache.org>,
	Vrushali Channapattan <vrushalic2016@gmail.com>,
	"common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>,
	"yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>,
	"mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hadoop.apache.org>,
	"hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>
Content-Type: multipart/alternative; boundary="001a114577ea08b12f055799d2c9"
archived-at: Fri, 25 Aug 2017 20:30:56 -0000

--001a114577ea08b12f055799d2c9
Content-Type: text/plain; charset="UTF-8"

+1 (binding).

I have been following the effort and had few design discussions around the
team especially about how it integrates with Federation. Overall I feel
it's a welcome improvement to YARN.

What are the timelines you are looking for getting this into branch-2?

Thanks,
Subru

On Fri, Aug 25, 2017 at 10:04 AM, Sangjin Lee <sjlee@apache.org> wrote:

> +1 (binding)
>
> I've built the current branch, and checked out a few basic areas including
> documentation. Also perused the most recent changes that went in.
>
> Thanks much for the great team work! I look forward to seeing it in action.
>
> Regards,
> Sangjin
>
> On Fri, Aug 25, 2017 at 9:27 AM, Haibo Chen <haibochen@cloudera.com>
> wrote:
>
> > +1 from my side.
> >
> > More from the perspective of ensuring there is no impact of ATSv2 when it
> > is off (by default), I deployed the latest YARN-5355 bits into a few
> > clusters and ran internal Smoke tests. The tests shows no impact when
> ATSv2
> > is off.
> >
> > Best,
> > Haibo
> >
> > On Thu, Aug 24, 2017 at 7:51 AM, Sunil G <sunilg@apache.org> wrote:
> >
> > > Thank you very much Vrushali, Rohith, Varun and other folks who made
> this
> > > happen. Great work, really appreciate the same!!
> > >
> > > +1 (binding) from my side:
> > >
> > > # Tested ATSv2 cluster in a secure cluster. Ran some basic jobs
> > > # Accessed new YARN UI which shows various flows/flow activity etc.
> Seems
> > > fine.
> > > # Based on code, looks like all apis are compatible.
> > > # REST api docs looks fine as well, I guess we could improve that a bit
> > > more post merge as well.
> > > # Adding to additional thoughts which are discussed here, native
> service
> > > also could publish events to atsv2. I think that work is also happened
> in
> > > branch.
> > >
> > > Looking forward to a much wider adoption of ATSv2 with more projects.
> > >
> > > Thanks
> > > Sunil
> > >
> > >
> > > On Tue, Aug 22, 2017 at 12:02 PM Vrushali Channapattan <
> > > vrushalic2016@gmail.com> wrote:
> > >
> > > > Hi folks,
> > > >
> > > > Per earlier discussion [1], I'd like to start a formal vote to merge
> > > > feature branch YARN-5355 [2] (Timeline Service v.2) to trunk. The
> vote
> > > will
> > > > run for 7 days, and will end August 29 11:00 PM PDT.
> > > >
> > > > We have previously completed one merge onto trunk [3] and Timeline
> > > Service
> > > > v2 has been part of Hadoop release 3.0.0-alpha1.
> > > >
> > > > Since then, we have been working on extending the capabilities of
> > > Timeline
> > > > Service v2 in a feature branch [2] for a while, and we are reasonably
> > > > confident that the state of the feature meets the criteria to be
> merged
> > > > onto trunk and we'd love folks to get their hands on it in a test
> > > capacity
> > > > and provide valuable feedback so that we can make it
> production-ready.
> > > >
> > > > In a nutshell, Timeline Service v.2 delivers significant scalability
> > and
> > > > usability improvements based on a new architecture. What we would
> like
> > to
> > > > merge to trunk is termed "alpha 2" (milestone 2). The feature has a
> > > > complete end-to-end read/write flow with security and read level
> > > > authorization via whitelists. You should be able to start setting it
> up
> > > and
> > > > testing it.
> > > >
> > > > At a high level, the following are the key features that have been
> > > > implemented since alpha1:
> > > > - Security via Kerberos Authentication and delegation tokens
> > > > - Read side simple authorization via whitelist
> > > > - Client configurable entity sort ordering
> > > > - Richer REST APIs for apps, app attempts, containers, fetching
> metrics
> > > by
> > > > timerange, pagination, sub-app entities
> > > > - Support for storing sub-application entities (entities that exist
> > > outside
> > > > the scope of an application)
> > > > - Configurable TTLs (time-to-live) for tables, configurable table
> > > prefixes,
> > > > configurable hbase cluster
> > > > - Flow level aggregations done as dynamic (table level) coprocessors
> > > > - Uses latest stable HBase release 1.2.6
> > > >
> > > > There are a total of 82 subtasks that were completed as part of this
> > > > effort.
> > > >
> > > > We paid close attention to ensure that once disabled Timeline Service
> > v.2
> > > > does not impact existing functionality when disabled (by default).
> > > >
> > > > Special thanks to a team of folks who worked hard and contributed
> > towards
> > > > this effort with patches, reviews and guidance: Rohith Sharma K S,
> > Varun
> > > > Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar Vavilapalli, Joep
> > > > Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal Stack.
> > > >
> > > > Regards,
> > > > Vrushali
> > > >
> > > > [1] http://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg27
> > 383.html
> > > > [2] https://issues.apache.org/jira/browse/YARN-5355
> > > > [3] https://issues.apache.org/jira/browse/YARN-2928
> > > > [4] https://github.com/apache/hadoop/commits/YARN-5355
> > > >
> > >
> >
>

--001a114577ea08b12f055799d2c9--