hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vrushali C <vrushalic2...@gmail.com>
Subject Re: [VOTE] Merge feature branch YARN-5355 (Timeline Service v2) to trunk
Date Thu, 07 Sep 2017 23:57:36 GMT
Thanks everyone.

It has been over a week (~9 days) since TSv2 has been merged to trunk with
no problems thus far. We are now thinking about merging timeline service v2
to branch2 some time in the next few weeks.

So far, we have been maintaining a branch2 based YARN-5355_branch2 along
with our trunk based feature branch YARN-5355. Varun Saxena has been
diligently rebasing it to stay current with branch2.

Currently we are in the process of testing it just like we did our due
diligence with the trunk based YARN-5355 branch and will ensure the TSv2
branch2 code is a stable state to be merged.

We will send out another email when we are ready to merge to branch2.
thanks
Vrushali

On Thu, Aug 31, 2017 at 12:33 PM, Subramaniam V K <subru.vk@gmail.com>
wrote:

> Good to see this merged. I have initiated a separate thread with a smaller
> set of stakeholders to discuss inclusion in 2.9. We'll report back to the
> 2.9 release thread as soon as we reach consensus.
>
> On Thu, Aug 31, 2017 at 10:39 AM, Ravi Prakash <ravihadoop@gmail.com>
> wrote:
>
>> +1 to maintaining history.
>>
>> On Wed, Aug 30, 2017 at 11:38 PM, varunsaxena@apache.org <
>> varun.saxena.apache@gmail.com> wrote:
>>
>> > Yes, I had used "git merge --no-ff"  while merging ATSv2 to trunk.
>> > Maintaining history I believe can be useful as it can make reverts
>> > easier if at all required.
>> > And can be an easy reference point to look at who had contributed what
>> > without having to go back to the branch.
>> >
>> > Regards,
>> > Varun Saxena.
>> >
>> > On Thu, Aug 31, 2017 at 3:56 AM, Vrushali C <vrushalic2016@gmail.com>
>> > wrote:
>> >
>> > > Thanks Sangjin for the link to the previous discussions on this! I
>> think
>> > > that helps answer Steve's questions.
>> > >
>> > > As decided on that thread [1], YARN-5355 as a feature branch was
>> merged
>> > to
>> > > trunk via "git merge --no-ff" .
>> > >
>> > > Although trunk already had TSv2 code (alpha1) prior to this merge, we
>> > > chose to develop on a feature branch YARN-5355 so that we could
>> control
>> > > when changes went into trunk and didn't inadvertently disrupt trunk.
>> > >
>> > > Is the latest merge causing any conflicts or issues for s3guard,
>> Steve?
>> > >
>> > > thanks
>> > > Vrushali
>> > > [1] https://lists.apache.org/thread.html/43cd65c6b6c3c0e8ac2b3c7
>> 6afd9ef
>> > > f1f78b177fabe9c4a96d9b3d0b@1440189889@%3Ccommon-dev.hadoop.apache.org
>> %3E
>> > >
>> > >
>> > > On Wed, Aug 30, 2017 at 2:37 PM, Sangjin Lee <sjlee@apache.org>
>> wrote:
>> > >
>> > >> I recall this discussion about a couple of years ago:
>> > >> https://lists.apache.org/thread.html/43cd65c6b6c3c0e8ac
>> > >> 2b3c76afd9eff1f78b177fabe9c4a96d9b3d0b@1440189889@%3Ccommon-
>> > >> dev.hadoop.apache.org%3E
>> > >>
>> > >> On Wed, Aug 30, 2017 at 2:32 PM, Steve Loughran <
>> stevel@hortonworks.com
>> > >
>> > >> wrote:
>> > >>
>> > >>> I'd have assumed it would have gone in as one single patch, rather
>> than
>> > >>> a full history. I don't see why the trunk needs all the evolutionary
>> > >>> history of a build.
>> > >>>
>> > >>> What should our policy/process be here?
>> > >>>
>> > >>> I do currently plan to merge the s3guard in as one single squashed
>> > >>> patch; just getting HADOOP-14809 sorted first.
>> > >>>
>> > >>>
>> > >>> > On 30 Aug 2017, at 07:09, Vrushali C <vrushalic2016@gmail.com>
>> > wrote:
>> > >>> >
>> > >>> > I'm adding my +1 (binding) to conclude the vote.
>> > >>> >
>> > >>> > With 13 +1's (11 binding) and no -1's, the vote passes. We'll
get
>> on
>> > >>> with
>> > >>> > the merge to trunk shortly. Thanks everyone!
>> > >>> >
>> > >>> > Regards
>> > >>> > Vrushali
>> > >>> >
>> > >>> >
>> > >>> > On Tue, Aug 29, 2017 at 10:54 AM, varunsaxena@apache.org <
>> > >>> > varun.saxena.apache@gmail.com> wrote:
>> > >>> >
>> > >>> >> +1 (binding).
>> > >>> >>
>> > >>> >> Kudos to all the team members for their great work!
>> > >>> >>
>> > >>> >> Being part of the ATSv2 team, I have been involved with
either
>> > >>> development
>> > >>> >> or review of most of the JIRAs'.
>> > >>> >> Tested ATSv2 in both secure and non-secure mode. Also
verified
>> that
>> > >>> there
>> > >>> >> is no impact when ATSv2 is turned off.
>> > >>> >>
>> > >>> >> Regards,
>> > >>> >> Varun Saxena.
>> > >>> >>
>> > >>> >> On Tue, Aug 22, 2017 at 12:02 PM, Vrushali Channapattan
<
>> > >>> >> vrushalic2016@gmail.com> wrote:
>> > >>> >>
>> > >>> >>> Hi folks,
>> > >>> >>>
>> > >>> >>> Per earlier discussion [1], I'd like to start a formal
vote to
>> > merge
>> > >>> >>> feature branch YARN-5355 [2] (Timeline Service v.2)
to trunk.
>> The
>> > >>> vote
>> > >>> >>> will
>> > >>> >>> run for 7 days, and will end August 29 11:00 PM PDT.
>> > >>> >>>
>> > >>> >>> We have previously completed one merge onto trunk
[3] and
>> Timeline
>> > >>> Service
>> > >>> >>> v2 has been part of Hadoop release 3.0.0-alpha1.
>> > >>> >>>
>> > >>> >>> Since then, we have been working on extending the
capabilities
>> of
>> > >>> Timeline
>> > >>> >>> Service v2 in a feature branch [2] for a while, and
we are
>> > reasonably
>> > >>> >>> confident that the state of the feature meets the
criteria to be
>> > >>> merged
>> > >>> >>> onto trunk and we'd love folks to get their hands
on it in a
>> test
>> > >>> capacity
>> > >>> >>> and provide valuable feedback so that we can make
it
>> > >>> production-ready.
>> > >>> >>>
>> > >>> >>> In a nutshell, Timeline Service v.2 delivers significant
>> > scalability
>> > >>> and
>> > >>> >>> usability improvements based on a new architecture.
What we
>> would
>> > >>> like to
>> > >>> >>> merge to trunk is termed "alpha 2" (milestone 2).
The feature
>> has a
>> > >>> >>> complete end-to-end read/write flow with security
and read level
>> > >>> >>> authorization via whitelists. You should be able to
start
>> setting
>> > it
>> > >>> up
>> > >>> >>> and
>> > >>> >>> testing it.
>> > >>> >>>
>> > >>> >>> At a high level, the following are the key features
that have
>> been
>> > >>> >>> implemented since alpha1:
>> > >>> >>> - Security via Kerberos Authentication and delegation
tokens
>> > >>> >>> - Read side simple authorization via whitelist
>> > >>> >>> - Client configurable entity sort ordering
>> > >>> >>> - Richer REST APIs for apps, app attempts, containers,
fetching
>> > >>> metrics by
>> > >>> >>> timerange, pagination, sub-app entities
>> > >>> >>> - Support for storing sub-application entities (entities
that
>> exist
>> > >>> >>> outside
>> > >>> >>> the scope of an application)
>> > >>> >>> - Configurable TTLs (time-to-live) for tables, configurable
>> table
>> > >>> >>> prefixes,
>> > >>> >>> configurable hbase cluster
>> > >>> >>> - Flow level aggregations done as dynamic (table level)
>> > coprocessors
>> > >>> >>> - Uses latest stable HBase release 1.2.6
>> > >>> >>>
>> > >>> >>> There are a total of 82 subtasks that were completed
as part of
>> > this
>> > >>> >>> effort.
>> > >>> >>>
>> > >>> >>> We paid close attention to ensure that once disabled
Timeline
>> > >>> Service v.2
>> > >>> >>> does not impact existing functionality when disabled
(by
>> default).
>> > >>> >>>
>> > >>> >>> Special thanks to a team of folks who worked hard
and
>> contributed
>> > >>> towards
>> > >>> >>> this effort with patches, reviews and guidance: Rohith
Sharma K
>> S,
>> > >>> Varun
>> > >>> >>> Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar
Vavilapalli,
>> > Joep
>> > >>> >>> Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal
Stack.
>> > >>> >>>
>> > >>> >>> Regards,
>> > >>> >>> Vrushali
>> > >>> >>>
>> > >>> >>> [1] http://www.mail-archive.com/ya
>> rn-dev@hadoop.apache.org/msg27
>> > >>> 383.html
>> > >>> >>> [2] https://issues.apache.org/jira/browse/YARN-5355
>> > >>> >>> [3] https://issues.apache.org/jira/browse/YARN-2928
>> > >>> >>> [4] https://github.com/apache/hadoop/commits/YARN-5355
>> > >>> >>>
>> > >>> >>
>> > >>> >>
>> > >>>
>> > >>>
>> > >>> ------------------------------------------------------------
>> ---------
>> > >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> > >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>> > >>>
>> > >>>
>> > >>
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message