hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brahma Reddy Battula <bra...@apache.org>
Subject [DISCUSS] Merge HDFS-13891(RBF) to trunk
Date Sun, 02 Jun 2019 03:40:13 GMT
Dear Hadoop Developers

I would like to propose RBF Branch (HDFS-13891) merge into trunk. We have
been working on this feature from last several months.
This feature work received the contributions from different companies. All
of the feature development happened smoothly and collaboratively in JIRAs.

Kindly do take a look at the branch and raise issues/concerns that need to
be addressed before the merge.

*Highlights of HDFS-13891 Branch:*

Adding Security to RBF(1)
Adding Missing Client API's(2)
Improvements/Bug Fixing
  Critical - HDFS-13637, HDFS-13834


No of JIRAs Resolved: 72

All this commits are in RBF Module. No changes in hdfs/common.

*Tested Cluster:*

Most of these changes verified at Uber,Microsoft,Huawei and some other

*Uber*: Most changes are running in production @Uber including the critical
security changes, HDFS Clusters are 4000+ nodes with 8 HDFS Routers.
Zookeeper as a state store to hold delegation tokens were also stress
tested to hold more than 2 Million tokens. --CR Hota

*MicroSoft*: Most of these changes are currently running in production at
Microsoft.The security has also been tested in a 500 server cluster with 4
subclsuters. --Inigo Goiri

*Huawei* : Deployed all this changes in 20 node cluster with 3
routers.Planning deploy 10K production cluster.


Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu Asanuma,Shubham
Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna Li,Gang
Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He Xiaoqiao,Sherwood
Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving discussions and
contributing to this.

*Future Tasks:*

 will cleanup the jira's under this umbrella and contiue to work.

1) https://issues.apache.org/jira/browse/HDFS-13532
2) https://issues.apache.org/jira/browse/HDFS-13655

--Brahma Reddy Battula

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message