hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jian He <...@hortonworks.com>
Subject Re: [VOTE] Merge feature branch YARN-5355 (Timeline Service v2) to trunk
Date Tue, 29 Aug 2017 05:47:28 GMT
+1 (binding)

I’ve participated in the review of ATSv2 security related patch.

JIan

> On Aug 28, 2017, at 1:39 AM, Rohith Sharma K S <rohithsharmaks@apache.org> wrote:
> 
> +1 (binding)
> 
> Thank you very much for the great team work!
> 
> Built from source and deployed in secured cluster. The below are the test
> result.
> 
> Deployment :
>    Standard hadoop security deployment authentication and authorization as
> well.
>    Branch-2 Hadoop and Hbase security cluster.
>    Branch-3 Hadoop security cluster. HBase client is pointing to Branch-2
> hbase cluster.
>    All security configurations are set in-place.
>    Each service is running with its own user. Say, HDFS is running with
> hdfs, YARN user is running with yarn, Hbase is running with hbase
>    Smoke test user : test-user
> 
> Test Cases :
> 
> Authentication :
> Verify for all daemons start up successful : OK
> Run a MR job using test-user : OK
> Verify for REST API’s with in the scope of application : OK
> Verify for REST API’s newly added I.e outside scope of application : OK.
> RM Restart/ NM restart / RM_work-preserving restart has executed and
> verified for data : OK. (Entity validation is done, but not entity data
> validation!
> Token redistribution to AM, NM is verified.
> 
> Authorization :
>     1 . Basic whitelisting of users to read has been validated. Works as
> expected!
> 
> Disabling TSv2 configuration is also being tested.
> 
> 
> Thanks & Regards
> Rohith Sharma K S
> 
> On 22 August 2017 at 12:02, Vrushali Channapattan <vrushalic2016@gmail.com>
> wrote:
> 
>> Hi folks,
>> 
>> Per earlier discussion [1], I'd like to start a formal vote to merge
>> feature branch YARN-5355 [2] (Timeline Service v.2) to trunk. The vote will
>> run for 7 days, and will end August 29 11:00 PM PDT.
>> 
>> We have previously completed one merge onto trunk [3] and Timeline Service
>> v2 has been part of Hadoop release 3.0.0-alpha1.
>> 
>> Since then, we have been working on extending the capabilities of Timeline
>> Service v2 in a feature branch [2] for a while, and we are reasonably
>> confident that the state of the feature meets the criteria to be merged
>> onto trunk and we'd love folks to get their hands on it in a test capacity
>> and provide valuable feedback so that we can make it production-ready.
>> 
>> In a nutshell, Timeline Service v.2 delivers significant scalability and
>> usability improvements based on a new architecture. What we would like to
>> merge to trunk is termed "alpha 2" (milestone 2). The feature has a
>> complete end-to-end read/write flow with security and read level
>> authorization via whitelists. You should be able to start setting it up and
>> testing it.
>> 
>> At a high level, the following are the key features that have been
>> implemented since alpha1:
>> - Security via Kerberos Authentication and delegation tokens
>> - Read side simple authorization via whitelist
>> - Client configurable entity sort ordering
>> - Richer REST APIs for apps, app attempts, containers, fetching metrics by
>> timerange, pagination, sub-app entities
>> - Support for storing sub-application entities (entities that exist outside
>> the scope of an application)
>> - Configurable TTLs (time-to-live) for tables, configurable table prefixes,
>> configurable hbase cluster
>> - Flow level aggregations done as dynamic (table level) coprocessors
>> - Uses latest stable HBase release 1.2.6
>> 
>> There are a total of 82 subtasks that were completed as part of this
>> effort.
>> 
>> We paid close attention to ensure that once disabled Timeline Service v.2
>> does not impact existing functionality when disabled (by default).
>> 
>> Special thanks to a team of folks who worked hard and contributed towards
>> this effort with patches, reviews and guidance: Rohith Sharma K S, Varun
>> Saxena, Haibo Chen, Sangjin Lee, Li Lu, Vinod Kumar Vavilapalli, Joep
>> Rottinghuis, Jason Lowe, Jian He, Robert Kanter, Micheal Stack.
>> 
>> Regards,
>> Vrushali
>> 
>> [1] http://www.mail-archive.com/yarn-dev@hadoop.apache.org/msg27383.html
>> [2] https://issues.apache.org/jira/browse/YARN-5355
>> [3] https://issues.apache.org/jira/browse/YARN-2928
>> [4] https://github.com/apache/hadoop/commits/YARN-5355
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

Mime
View raw message