hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akira Ajisaka <aajis...@apache.org>
Subject Re: [VOTE] Merge HDFS-13891(RBF) to trunk
Date Mon, 10 Jun 2019 02:54:48 GMT
+1

-Akira

On Sun, Jun 9, 2019 at 10:56 PM Brahma Reddy Battula <brahma@apache.org> wrote:
>
> Hi All,
>
>     Given the positive response to the discussion thread [1], here is the
> formal vote thread to merge HDFS-13891 in to trunk.
>
>     Summary of code changes:
>     1. Code changes for this branch are done in the hadoop-hdfs-rbf
> subproject, there is no impact to hadoop-hdfs and hadoop-common.
>     2. Added Security support for RBF
>     3. Added Missing Client Protocol API's
>     4. Bug Fixes/ Improvments
>
>
>      The vote will run for 7 days, ending Sat June 15th. I will start this
> vote with my +1.
>
> Regards,
> Brahma Reddy Battula
>
> 1).
> https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E
>
>
> On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He <xq.he2009@gmail.com> wrote:
>
> > Thanks Brahma for starting the thread.
> > +1 for merging.
> >
> > He Xiaoqiao
> >
> > On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota <chitts.hota@gmail.com>
> > wrote:
> >
> >> Thanks Brahma initiating this.
> >> +1(non-binding) for merge.
> >>
> >> @Uber we have almost all changes specially rbf security in production for
> >> a
> >> while now without issues.
> >>
> >> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola <
> >> giovanni.fumarola@gmail.com> wrote:
> >>
> >> > +1 on merging.
> >> >
> >> > Thanks Brahma for starting the thread.
> >> >
> >> > On Mon, Jun 3, 2019 at 10:00 AM IƱigo Goiri <elgoiri@gmail.com>
wrote:
> >> >
> >> > > Thank you Brahma for pushing this.
> >> > >
> >> > > As you mentioned, we have already taken most of the changes into
> >> > > production.
> >> > > I want to highlight that the main contribution is the addition of
> >> > security.
> >> > > We have been able to test this at a smaller scale (~500 servers and
4
> >> > > subclusters) and the performance is great with our current ZooKeeper
> >> > > deployment.
> >> > > I would also like to highlight that all the changes are constrained
to
> >> > > hadoop-hdfs-rbf and there is no differences in commons or HDFS.
> >> > >
> >> > > +1 on merging
> >> > >
> >> > > Inigo
> >> > >
> >> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka <aajisaka@gmail.com>
> >> > wrote:
> >> > >
> >> > > > Thanks Brahma for starting the discussion.
> >> > > > I'm +1 for merging this.
> >> > > >
> >> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes
cluster
> >> > > > with 2 routers (not in production) and running several tests.
> >> > > >
> >> > > > Regards,
> >> > > > Akira
> >> > > >
> >> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula <
> >> > brahma@apache.org>
> >> > > > wrote:
> >> > > > >
> >> > > > > Dear Hadoop Developers
> >> > > > >
> >> > > > > I would like to propose RBF Branch (HDFS-13891) merge into
trunk.
> >> We
> >> > > have
> >> > > > > been working on this feature from last several months.
> >> > > > > This feature work received the contributions from different
> >> > companies.
> >> > > > All
> >> > > > > of the feature development happened smoothly and collaboratively
> >> in
> >> > > > JIRAs.
> >> > > > >
> >> > > > > Kindly do take a look at the branch and raise issues/concerns
that
> >> > need
> >> > > > to
> >> > > > > be addressed before the merge.
> >> > > > >
> >> > > > > *Highlights of HDFS-13891 Branch:*
> >> > > > > =============================
> >> > > > >
> >> > > > > Adding Security to RBF(1)
> >> > > > > Adding Missing Client API's(2)
> >> > > > > Improvements/Bug Fixing
> >> > > > >   Critical - HDFS-13637, HDFS-13834
> >> > > > >
> >> > > > > *Commits:*
> >> > > > > ========
> >> > > > >
> >> > > > > No of JIRAs Resolved: 72
> >> > > > >
> >> > > > > All this commits are in RBF Module. No changes in hdfs/common.
> >> > > > >
> >> > > > > *Tested Cluster:*
> >> > > > > =============
> >> > > > >
> >> > > > > Most of these changes verified at Uber,Microsoft,Huawei
and some
> >> > other
> >> > > > > compaines.
> >> > > > >
> >> > > > > *Uber*: Most changes are running in production @Uber including
the
> >> > > > critical
> >> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS
> >> Routers.
> >> > > > > Zookeeper as a state store to hold delegation tokens were
also
> >> stress
> >> > > > > tested to hold more than 2 Million tokens. --CR Hota
> >> > > > >
> >> > > > > *MicroSoft*: Most of these changes are currently running
in
> >> > production
> >> > > at
> >> > > > > Microsoft.The security has also been tested in a 500 server
> >> cluster
> >> > > with
> >> > > > 4
> >> > > > > subclsuters. --Inigo Goiri
> >> > > > >
> >> > > > > *Huawei* : Deployed all this changes in 20 node cluster
with 3
> >> > > > > routers.Planning deploy 10K production cluster.
> >> > > > >
> >> > > > > *Contributors:*
> >> > > > > ===========
> >> > > > >
> >> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu
> >> Asanuma,Shubham
> >> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna
Li,Gang
> >> > > > > Li,Surendra Singh Lihore,Ranith Sardar,Ayush Saxena,He
> >> > > Xiaoqiao,Sherwood
> >> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving
> >> > discussions
> >> > > > and
> >> > > > > contributing to this.
> >> > > > >
> >> > > > > *Future Tasks:*
> >> > > > > ============
> >> > > > >
> >> > > > >  will cleanup the jira's under this umbrella and contiue
to work.
> >> > > > >
> >> > > > > Reference:
> >> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532
> >> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > --Brahma Reddy Battula
> >> > > >
> >> > > >
> >> ---------------------------------------------------------------------
> >> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> > > > For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >> --
> >> Regards,
> >> Chitta
> >>
> >
>
> --
>
>
>
> --Brahma Reddy Battula
> --
>
>
>
> --Brahma Reddy Battula

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


Mime
View raw message