hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vrushali C <vrushalic2...@gmail.com>
Subject Re: [VOTE] Merge feature branch YARN-5355 (Timeline Service v2) to trunk
Date Mon, 16 Oct 2017 16:24:04 GMT
Timeline Service v2 should be landing on branch2 shortly.

thanks
Vrushali

On Thu, Sep 7, 2017 at 4:57 PM, Vrushali C <vrushalic2016@gmail.com> wrote:

> Thanks everyone.
>
> It has been over a week (~9 days) since TSv2 has been merged to trunk with
> no problems thus far. We are now thinking about merging timeline service v2
> to branch2 some time in the next few weeks.
>
> So far, we have been maintaining a branch2 based YARN-5355_branch2 along
> with our trunk based feature branch YARN-5355. Varun Saxena has been
> diligently rebasing it to stay current with branch2.
>
> Currently we are in the process of testing it just like we did our due
> diligence with the trunk based YARN-5355 branch and will ensure the TSv2
> branch2 code is a stable state to be merged.
>
> We will send out another email when we are ready to merge to branch2.
> thanks
> Vrushali
>
> On Thu, Aug 31, 2017 at 12:33 PM, Subramaniam V K <subru.vk@gmail.com>
> wrote:
>
>> Good to see this merged. I have initiated a separate thread with a
>> smaller set of stakeholders to discuss inclusion in 2.9. We'll report back
>> to the 2.9 release thread as soon as we reach consensus.
>>
>> On Thu, Aug 31, 2017 at 10:39 AM, Ravi Prakash <ravihadoop@gmail.com>
>> wrote:
>>
>>> +1 to maintaining history.
>>>
>>> On Wed, Aug 30, 2017 at 11:38 PM, varunsaxena@apache.org <
>>> varun.saxena.apache@gmail.com> wrote:
>>>
>>> > Yes, I had used "git merge --no-ff"  while merging ATSv2 to trunk.
>>> > Maintaining history I believe can be useful as it can make reverts
>>> > easier if at all required.
>>> > And can be an easy reference point to look at who had contributed what
>>> > without having to go back to the branch.
>>> >
>>> > Regards,
>>> > Varun Saxena.
>>> >
>>> > On Thu, Aug 31, 2017 at 3:56 AM, Vrushali C <vrushalic2016@gmail.com>
>>> > wrote:
>>> >
>>> > > Thanks Sangjin for the link to the previous discussions on this! I
>>> think
>>> > > that helps answer Steve's questions.
>>> > >
>>> > > As decided on that thread [1], YARN-5355 as a feature branch was
>>> merged
>>> > to
>>> > > trunk via "git merge --no-ff" .
>>> > >
>>> > > Although trunk already had TSv2 code (alpha1) prior to this merge,
we
>>> > > chose to develop on a feature branch YARN-5355 so that we could
>>> control
>>> > > when changes went into trunk and didn't inadvertently disrupt trunk.
>>> > >
>>> > > Is the latest merge causing any conflicts or issues for s3guard,
>>> Steve?
>>> > >
>>> > > thanks
>>> > > Vrushali
>>> > > [1] https://lists.apache.org/thread.html/43cd65c6b6c3c0e8ac2b3c7
>>> 6afd9ef
>>> > > f1f78b177fabe9c4a96d9b3d0b@1440189889@%3Ccommon-dev.hadoop.a
>>> pache.org%3E
>>> > >
>>> > >
>>> > > On Wed, Aug 30, 2017 at 2:37 PM, Sangjin Lee <sjlee@apache.org>
>>> wrote:
>>> > >
>>> > >> I recall this discussion about a couple of years ago:
>>> > >> https://lists.apache.org/thread.html/43cd65c6b6c3c0e8ac
>>> > >> 2b3c76afd9eff1f78b177fabe9c4a96d9b3d0b@1440189889@%3Ccommon-
>>> > >> dev.hadoop.apache.org%3E
>>> > >>
>>> > >> On Wed, Aug 30, 2017 at 2:32 PM, Steve Loughran <
>>> stevel@hortonworks.com
>>> > >
>>> > >> wrote:
>>> > >>
>>> > >>> I'd have assumed it would have gone in as one single patch,
rather
>>> than
>>> > >>> a full history. I don't see why the trunk needs all the
>>> evolutionary
>>> > >>> history of a build.
>>> > >>>
>>> > >>> What should our policy/process be here?
>>> > >>>
>>> > >>> I do currently plan to merge the s3guard in as one single squashed
>>> > >>> patch; just getting HADOOP-14809 sorted first.
>>> > >>>
>>> > >>>
>>> > >>> > On 30 Aug 2017, at 07:09, Vrushali C <vrushalic2016@gmail.com>
>>> > wrote:
>>> > >>> >
>>> > >>> > I'm adding my +1 (binding) to conclude the vote.
>>> > >>> >
>>> > >>> > With 13 +1's (11 binding) and no -1's, the vote passes.
We'll
>>> get on
>>> > >>> with
>>> > >>> > the merge to trunk shortly. Thanks everyone!
>>> > >>> >
>>> > >>> > Regards
>>> > >>> > Vrushali
>>> > >>> >
>>> > >>> >
>>> > >>> > On Tue, Aug 29, 2017 at 10:54 AM, varunsaxena@apache.org
<
>>> > >>> > varun.saxena.apache@gmail.com> wrote:
>>> > >>> >
>>> > >>> >> +1 (binding).
>>> > >>> >>
>>> > >>> >> Kudos to all the team members for their great work!
>>> > >>> >>
>>> > >>> >> Being part of the ATSv2 team, I have been involved
with either
>>> > >>> development
>>> > >>> >> or review of most of the JIRAs'.
>>> > >>> >> Tested ATSv2 in both secure and non-secure mode. Also
verified
>>> that
>>> > >>> there
>>> > >>> >> is no impact when ATSv2 is turned off.
>>> > >>> >>
>>> > >>> >> Regards,
>>> > >>> >> Varun Saxena.
>>> > >>> >>
>>> > >>> >> On Tue, Aug 22, 2017 at 12:02 PM, Vrushali Channapattan
<
>>> > >>> >> vrushalic2016@gmail.com> wrote:
>>> > >>> >>
>>> > >>> >>> Hi folks,
>>> > >>> >>>
>>> > >>> >>> Per earlier discussion [1], I'd like to start
a formal vote to
>>> > merge
>>> > >>> >>> feature branch YARN-5355 [2] (Timeline Service
v.2) to trunk.
>>> The
>>> > >>> vote
>>> > >>> >>> will
>>> > >>> >>> run for 7 days, and will end August 29 11:00 PM
PDT.
>>> > >>> >>>
>>> > >>> >>> We have previously completed one merge onto trunk
[3] and
>>> Timeline
>>> > >>> Service
>>> > >>> >>> v2 has been part of Hadoop release 3.0.0-alpha1.
>>> > >>> >>>
>>> > >>> >>> Since then, we have been working on extending
the capabilities
>>> of
>>> > >>> Timeline
>>> > >>> >>> Service v2 in a feature branch [2] for a while,
and we are
>>> > reasonably
>>> > >>> >>> confident that the state of the feature meets
the criteria to
>>> be
>>> > >>> merged
>>> > >>> >>> onto trunk and we'd love folks to get their hands
on it in a
>>> test
>>> > >>> capacity
>>> > >>> >>> and provide valuable feedback so that we can make
it
>>> > >>> production-ready.
>>> > >>> >>>
>>> > >>> >>> In a nutshell, Timeline Service v.2 delivers significant
>>> > scalability
>>> > >>> and
>>> > >>> >>> usability improvements based on a new architecture.
What we
>>> would
>>> > >>> like to
>>> > >>> >>> merge to trunk is termed "alpha 2" (milestone
2). The feature
>>> has a
>>> > >>> >>> complete end-to-end read/write flow with security
and read
>>> level
>>> > >>> >>> authorization via whitelists. You should be able
to start
>>> setting
>>> > it
>>> > >>> up
>>> > >>> >>> and
>>> > >>> >>> testing it.
>>> > >>> >>>
>>> > >>> >>> At a high level, the following are the key features
that have
>>> been
>>> > >>> >>> implemented since alpha1:
>>> > >>> >>> - Security via Kerberos Authentication and delegation
tokens
>>> > >>> >>> - Read side simple authorization via whitelist
>>> > >>> >>> - Client configurable entity sort ordering
>>> > >>> >>> - Richer REST APIs for apps, app attempts, containers,
fetching
>>> > >>> metrics by
>>> > >>> >>> timerange, pagination, sub-app entities
>>> > >>> >>> - Support for storing sub-application entities
(entities that
>>> exist
>>> > >>> >>> outside
>>> > >>> >>> the scope of an application)
>>> > >>> >>> - Configurable TTLs (time-to-live) for tables,
configurable
>>> table
>>> > >>> >>> prefixes,
>>> > >>> >>> configurable hbase cluster
>>> > >>> >>> - Flow level aggregations done as dynamic (table
level)
>>> > coprocessors
>>> > >>> >>> - Uses latest stable HBase release 1.2.6
>>> > >>> >>>
>>> > >>> >>> There are a total of 82 subtasks that were completed
as part of
>>> > this
>>> > >>> >>> effort.
>>> > >>> >>>
>>> > >>> >>> We paid close attention to ensure that once disabled
Timeline
>>> > >>> Service v.2
>>> > >>> >>> does not impact existing functionality when disabled
(by
>>> default).
>>> > >>> >>>
>>> > >>> >>> Special thanks to a team of folks who worked hard
and
>>> contributed
>>> > >>> towards
>>> > >>> >>> this effort with patches, reviews and guidance:
Rohith Sharma
>>> K S,
>>> > >>> Varun
>>> > >>> >>> Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod
Kumar
>>> Vavilapalli,
>>> > Joep
>>> > >>> >>> Rottinghuis, Jason Lowe, Jian He, Robert Kanter,
Micheal Stack.
>>> > >>> >>>
>>> > >>> >>> Regards,
>>> > >>> >>> Vrushali
>>> > >>> >>>
>>> > >>> >>> [1] http://www.mail-archive.com/ya
>>> rn-dev@hadoop.apache.org/msg27
>>> > >>> 383.html
>>> > >>> >>> [2] https://issues.apache.org/jira/browse/YARN-5355
>>> > >>> >>> [3] https://issues.apache.org/jira/browse/YARN-2928
>>> > >>> >>> [4] https://github.com/apache/hadoop/commits/YARN-5355
>>> > >>> >>>
>>> > >>> >>
>>> > >>> >>
>>> > >>>
>>> > >>>
>>> > >>> ------------------------------------------------------------
>>> ---------
>>> > >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> > >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>> > >>>
>>> > >>>
>>> > >>
>>> > >
>>> >
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message