hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chittaranjan Hota <chitts.h...@gmail.com>
Subject Re: [DISCUSS] Merge HDFS-13891(RBF) to trunk
Date Tue, 04 Jun 2019 05:53:39 GMT
Thanks Brahma initiating this.
+1(non-binding) for merge.

@Uber we have almost all changes specially rbf security in production for a
while now without issues.

On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
giovanni.fumarola@gmail.com> wrote:

> +1 on merging.
>
> Thanks Brahma for starting the thread.
>
> On Mon, Jun 3, 2019 at 10:00 AM IƱigo Goiri <elgoiri@gmail.com> wrote:
>
> > Thank you Brahma for pushing this.
> >
> > As you mentioned, we have already taken most of the changes into
> > production.
> > I want to highlight that the main contribution is the addition of
> security.
> > We have been able to test this at a smaller scale (~500 servers and 4
> > subclusters) and the performance is great with our current ZooKeeper
> > deployment.
> > I would also like to highlight that all the changes are constrained to
> > hadoop-hdfs-rbf and there is no differences in commons or HDFS.
> >
> > +1 on merging
> >
> > Inigo
> >
> > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aajisaka@gmail.com>
> wrote:
> >
> > > Thanks Brahma for starting the discussion.
> > > I'm +1 for merging this.
> > >
> > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes cluster
> > > with 2 routers (not in production) and running several tests.
> > >
> > > Regards,
> > > Akira
> > >
> > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
> brahma@apache.org>
> > > wrote:
> > > >
> > > > Dear Hadoop Developers
> > > >
> > > > I would like to propose RBF Branch (HDFS-13891) merge into trunk. We
> > have
> > > > been working on this feature from last several months.
> > > > This feature work received the contributions from different
> companies.
> > > All
> > > > of the feature development happened smoothly and collaboratively in
> > > JIRAs.
> > > >
> > > > Kindly do take a look at the branch and raise issues/concerns that
> need
> > > to
> > > > be addressed before the merge.
> > > >
> > > > *Highlights of HDFS-13891 Branch:*
> > > > =============================
> > > >
> > > > Adding Security to RBF(1)
> > > > Adding Missing Client API's(2)
> > > > Improvements/Bug Fixing
> > > >   Critical - HDFS-13637, HDFS-13834
> > > >
> > > > *Commits:*
> > > > ========
> > > >
> > > > No of JIRAs Resolved: 72
> > > >
> > > > All this commits are in RBF Module. No changes in hdfs/common.
> > > >
> > > > *Tested Cluster:*
> > > > =============
> > > >
> > > > Most of these changes verified at Uber,Microsoft,Huawei and some
> other
> > > > compaines.
> > > >
> > > > *Uber*: Most changes are running in production @Uber including the
> > > critical
> > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS Routers.
> > > > Zookeeper as a state store to hold delegation tokens were also stress
> > > > tested to hold more than 2 Million tokens. --CR Hota
> > > >
> > > > *MicroSoft*: Most of these changes are currently running in
> production
> > at
> > > > Microsoft.The security has also been tested in a 500 server cluster
> > with
> > > 4
> > > > subclsuters. --Inigo Goiri
> > > >
> > > > *Huawei* : Deployed all this changes in 20 node cluster with 3
> > > > routers.Planning deploy 10K production cluster.
> > > >
> > > > *Contributors:*
> > > > ===========
> > > >
> > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu Asanuma,Shubham
> > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna Li,Gang
> > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He
> > Xiaoqiao,Sherwood
> > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
> discussions
> > > and
> > > > contributing to this.
> > > >
> > > > *Future Tasks:*
> > > > ============
> > > >
> > > >  will cleanup the jira's under this umbrella and contiue to work.
> > > >
> > > > Reference:
> > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
> > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
> > > >
> > > >
> > > >
> > > >
> > > > --Brahma Reddy Battula
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> > >
> > >
> >
>


-- 
Regards,
Chitta

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message