hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Wang <andrew.w...@cloudera.com>
Subject Re: [DISCUSS] Merging YARN-5355 (Timeline Service v.2) to trunk
Date Wed, 16 Aug 2017 18:44:40 GMT
Great, thanks Vrushali! Sounds good to me.

I have a few procedural release notes comments I'll put on YARN-5355, to
make sure we advertise this to our users appropriately.

On Wed, Aug 16, 2017 at 11:32 AM, Vrushali Channapattan <
vrushali.c@gmail.com> wrote:

> Hi Andrew,
>
> Thanks for your response!
>
> There have been no changes to existing APIs since alpha1.
>
> We at Twitter have tested the feature to demonstrate it works at what we
> consider moderate scale but this did not include the security related
> testing. The security testing is in progress at present by Timeline Service
> V2 team in the community and we think we will have more details on this
> very soon.
>
> About the jiras under YARN-5355: Only 3 of those sub-tasks are what we
> think of as "merge-blockers". The issues being targeted for merge are in
> [link1] below. There are about 59 jiras of which 56 are completed.
>
> We plan to make a new umbrella jira after the merge to trunk. We will then
> create a new branch with the new jira name and move these open jiras under
> YARN-5355 as subtasks of that new umbrella jira.
>
> thanks
> Vrushali
> [link1] https://issues.apache.org/jira/projects/YARN/versions/12337991
>
>
> On Wed, Aug 16, 2017 at 10:47 AM, Andrew Wang <andrew.wang@cloudera.com>
> wrote:
>
>> Hi Vrushali,
>>
>> Glad to hear this major dev milestone is nearing completion!
>>
>> Repeating my request on other merge [DISCUSS] threads, could you comment
>> on testing and API stability of this merge? Our timeline for beta1 is about
>> a month out, so there's not much time to fix things beforehand.
>>
>> Looking at YARN-5355 there are also many unresolved subtasks. Should most
>> of these be moved out to a new umbrella? I'm wondering what needs to be
>> completed before sending the merge vote.
>>
>> Given that TSv2 is committed for 3.0.0 GA, I'm more willing to flex the
>> beta1 release date for this feature than others. Hopefully that won't be
>> necessary though :)
>>
>> Best,
>> Andrew
>>
>> On Wed, Aug 16, 2017 at 10:26 AM, Vrushali Channapattan <
>> vrushalic2016@gmail.com> wrote:
>>
>>> Looks like some of the hyperlinks appear messed up, my apologies,
>>> resending
>>> the same email with hopefully better looking content:
>>>
>>> Hi All,
>>>
>>> I'd like to open a discussion for merging Timeline Service v2 (YARN-5355)
>>> to trunk in a few weeks.
>>>
>>> We have previously completed one merge onto trunk [1] and Timeline
>>> Service
>>> v2 has been part of Hadoop release 3.0.0-alpha1.
>>>
>>> Since then, we have been working on extending the capabilities of
>>> Timeline
>>> Service v2 in a feature branch [2].  There are a few related issues
>>> pending
>>> that are being actively worked upon and tested. As soon as they are
>>> resolved, we plan on starting a merge vote within the next two weeks. The
>>> goal is to get this into hadoop3 beta.
>>>
>>> We have paid close attention to ensure that  once disabled Timeline
>>> Service
>>> v2 does not impact existing functionality when disabled (by default).
>>>
>>> At a high level, following are the key features that have been
>>> implemented
>>> since 3.0.0-alpha1:
>>> - Security (via Kerberos Authentication & delegation tokens) [YARN-3053]
>>> - Timeline server usability improvements [timeline-server
>>> <https://issues.apache.org/jira/browse/YARN-5715?jql=project
>>> %20%3D%20YARN%20AND%20fixVersion%20%3D%20YARN-5355%20AND%20c
>>> omponent%20%3D%20timelineserver%20AND%20labels%20!%3D%20atsv
>>> 2-hbase%20ORDER%20BY%20updated%20ASC%2C%20priority%20DESC%
>>> 2C%20created%20ASC>
>>> ]
>>> - HBase specific improvements [atsv2-hbase
>>> <https://issues.apache.org/jira/browse/YARN-6604?jql=project
>>> %20%3D%20YARN%20AND%20fixVersion%20%3D%20YARN-5355%20AND%20l
>>> abels%20%3D%20atsv2-hbase%20ORDER%20BY%20updated%20DESC%2C%
>>> 20affectedVersion%20DESC%2C%20priority%20DESC%2C%20created%20ASC>
>>> ]
>>> - REST API additions and improvements [timeline-reader
>>> <https://issues.apache.org/jira/browse/YARN-5739?jql=project
>>> %20%3D%20YARN%20AND%20fixVersion%20%3D%20YARN-5355%20AND%20c
>>> omponent%20in%20(timelineclient%2C%20timelinereader)%
>>> 20ORDER%20BY%20updated%20ASC%2C%20priority%20DESC%2C%20created%20ASC>
>>> ]
>>> - Reader side simple authorization via whitelist [YARN-6820]
>>>
>>> We would love to get your thoughts on this before we open a real voting
>>> thread.
>>>
>>> Special thanks to a team of folks who worked hard and contributed towards
>>> this effort via patches, reviews and guidance: Rohith Sharma K S, Varun
>>> Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar Vavilapalli, Joep
>>> Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal Stack.
>>>
>>> Thanks
>>> Vrushali
>>> [1] Merge to trunk: http://www.mail-archive.com/yarn-dev@hadoop.apache.
>>> org/msg23897.html
>>> <http://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg23897.html>
>>> [2] feature branch YARN-5355 commits: https://github.com/ap
>>> ache/hadoop/commits/YARN-5355
>>>
>>> <https://github.com/apache/hadoop/commits/YARN-5355>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Aug 16, 2017 at 10:02 AM, Vrushali Channapattan <
>>> vrushali.c@gmail.com> wrote:
>>>
>>> > Hi All
>>> >
>>> > I’d like to open a discussion for merging Timeline Service v.2
>>> (YARN-5355)
>>> > to trunk in a few weeks.
>>> >
>>> > We have previously completed one merge onto trunk [1] and Timeline
>>> Service
>>> > v2 has been part of Hadoop release 3.0.0-alpha1.
>>> >
>>> > Since then, we have been working on extending the capabilities of
>>> Timeline
>>> > Service v2 in a feature branch [2]. There are a few related issues
>>> pending
>>> > that are being actively worked upon & tested. As soon as they are
>>> resolved,
>>> > we plan on starting a merge vote within the next 2 weeks. The goal is
>>> to
>>> > get this in for Hadoop3 beta.
>>> >
>>> > We have paid close attention to ensure that once disabled Timeline
>>> Service
>>> > v.2 does not impact existing functionality when disabled (by default).
>>> >
>>> > At a high level, following are the key features that have been
>>> implemented
>>> > since 3.0.0-alpha1:
>>> > - Security (via Kerberos Authentication & delegation tokens) at the
>>> writer
>>> > [YARN-3053]
>>> > - Timeline server usability improvements [timeline-server
>>> > <https://issues.apache.org/jira/browse/YARN-5715?jql=
>>> > project%20%3D%20YARN%20AND%20fixVersion%20%3D%20YARN-
>>> > 5355%20AND%20component%20%3D%20timelineserver%20AND%
>>> > 20labels%20%21%3D%20atsv2-hbase%20ORDER%20BY%20updated%
>>> > 20ASC%2C%20priority%20DESC%2C%20created%20ASC>
>>> > ]
>>> > - HBase specific improvements [atsv2-hbase
>>> > <https://issues.apache.org/jira/browse/YARN-6604?jql=
>>> > project%20%3D%20YARN%20AND%20fixVersion%20%3D%20YARN-
>>> > 5355%20AND%20labels%20%3D%20atsv2-hbase%20ORDER%20BY%20updat
>>> ed%20DESC%2C%
>>> > 20affectedVersion%20DESC%2C%20priority%20DESC%2C%20created%20ASC>
>>> > ]
>>> > - REST API additions & improvements [timeline-reader
>>> > <https://issues.apache.org/jira/browse/YARN-5739?jql=
>>> > project%20%3D%20YARN%20AND%20fixVersion%20%3D%20YARN-
>>> > 5355%20AND%20component%20in%20%28timelineclient%2C%
>>> > 20timelinereader%29%20ORDER%20BY%20updated%20ASC%2C%20priori
>>> ty%20DESC%2C%
>>> > 20created%20ASC>
>>> > ]
>>> > - Reader side simple authorization via whitelist [YARN-6820]
>>> >
>>> > We would love to get your thoughts on these and more before we open a
>>> real
>>> > voting thread.
>>> >
>>> > Special thanks to a team of folks who worked hard and contributed
>>> towards
>>> > this effort with patches, reviews and guidance: Rohith Sharma K S,
>>> Varun
>>> > Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar Vavilapalli, Joep
>>> > Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal Stack.
>>> >
>>> > Thanks!
>>> > Vrushali
>>> >
>>> > [1] Merge to trunk: http://www.mail-archive.com/yarn-dev@hadoop.
>>> > apache.org/msg23897.html
>>> > [2] feature branch YARN-5355 commits: https://github.com/
>>> > apache/hadoop/commits/YARN-5355
>>> >
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message