From hdfs-dev-return-37061-archive-asf-public=cust-asf.ponee.io@hadoop.apache.org Sun Jun 16 19:28:27 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 388E7180662 for ; Sun, 16 Jun 2019 21:28:27 +0200 (CEST) Received: (qmail 57204 invoked by uid 500); 16 Jun 2019 19:28:23 -0000 Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list hdfs-dev@hadoop.apache.org Received: (qmail 57182 invoked by uid 99); 16 Jun 2019 19:28:22 -0000 Received: from Unknown (HELO mailrelay1-lw-us.apache.org) (10.10.3.159) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 16 Jun 2019 19:28:22 +0000 Received: from mail-vs1-f43.google.com (mail-vs1-f43.google.com [209.85.217.43]) by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id 3C76B8BD1; Sun, 16 Jun 2019 19:28:22 +0000 (UTC) Received: by mail-vs1-f43.google.com with SMTP id q64so4859363vsd.1; Sun, 16 Jun 2019 12:28:22 -0700 (PDT) X-Gm-Message-State: APjAAAUAaJxT0gSq4LK6uAtDAEExDygvo0e80m0cRhwWyR6wdVNhLf4f LML6SQeyl0qB/Q8P8wp7a4NDk9g28BRKTJtE7/4= X-Google-Smtp-Source: APXvYqzmcHPMjrq+SSsh0VLF8Th5MBRK9836NqQdYFuSsRFCam4TPFyPSCB+KMMrxerzM3Lzpp4kwUjeOzZJpdivm9Y= X-Received: by 2002:a67:ed49:: with SMTP id m9mr59274704vsp.32.1560713301899; Sun, 16 Jun 2019 12:28:21 -0700 (PDT) MIME-Version: 1.0 References: <215DD061CF3CA0458E2665D5A0AB5142011D70AB@BLREML504-MBB.china.huawei.com> In-Reply-To: From: Brahma Reddy Battula Date: Mon, 17 Jun 2019 00:58:10 +0530 X-Gmail-Original-Message-ID: Message-ID: Subject: [RESULT][VOTE] Merge HDFS-13891(RBF) to trunk To: Takanobu Asanuma Cc: Hadoop Common , Hdfs-dev , Xiaoqiao He Content-Type: multipart/alternative; boundary="000000000000f4c000058b75e1e8" --000000000000f4c000058b75e1e8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Vote is passed with 3 binding +1's and no -1 ( 5 nonbinding votes) . thanks to all for voting. Will try to merge this week.. On Tue, Jun 11, 2019 at 1:13 PM Takanobu Asanuma wrote: > +1(non-binding). > > Regards, > - Takanobu > > ________________________________________ > From: Xiaoqiao He > Sent: Monday, June 10, 2019 15:47 > To: Ranith Sardar > Cc: Brahma Reddy Battula; Hadoop Common; Hdfs-dev > Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk > > +1 (non-binding) > > - Try to merge branch HDFS-13891(RBF) to trunk at local and no conflict o= r > failure. > - Built from merged sources. > - Ran 85 RBF test class at local and result shows: (Tests run: 639, > Failures: 2, Errors: 2, Skipped: 2) - failed tests include > #TestRouterWithSecureStartup and #TestRouterHttpDelegationToken. I don't > think it is blocked issue. > > Thanks Brahma for organizing the merge. > > > On Mon, Jun 10, 2019 at 12:27 PM Ranith Sardar > wrote: > > > +1 (Non-binding) > > > > -----Original Message----- > > From: Brahma Reddy Battula [mailto:brahma@apache.org] > > Sent: 09 June 2019 19:31 > > To: Hadoop Common ; Hdfs-dev < > > hdfs-dev@hadoop.apache.org> > > Subject: [VOTE] Merge HDFS-13891(RBF) to trunk > > > > Updated mail... > > > > ---------- Forwarded message --------- > > From: Brahma Reddy Battula > > Date: Sun, Jun 9, 2019 at 7:26 PM > > Subject: Re: [VOTE] Merge HDFS-13891(RBF) to trunk > > To: Xiaoqiao He > > Cc: Akira Ajisaka , Chittaranjan Hota < > > chitts.hota@gmail.com>, Giovanni Matteo Fumarola < > > giovanni.fumarola@gmail.com>, Hadoop Common < > common-dev@hadoop.apache.org>, > > Hdfs-dev , I=C3=B1igo Goiri > > > > > > Hi All, > > > > Given the positive response to the discussion thread [1], here is t= he > > formal vote thread to merge HDFS-13891 in to trunk. > > > > Summary of code changes: > > 1. Code changes for this branch are done in the hadoop-hdfs-rbf > > subproject, there is no impact to hadoop-hdfs and hadoop-common. > > 2. Added Security support for RBF > > 3. Added Missing Client Protocol API's > > 4. Bug Fixes/ Improvments > > > > > > The vote will run for 7 days, ending Sat June 15th. I will start > this > > vote with my +1. > > > > Regards, > > Brahma Reddy Battula > > > > 1). > > > > > https://lists.apache.org/thread.html/cdc2e084874b30bf6af2dd827bcbdba4ab8d= 3d983a8b9796e61608e6@%3Ccommon-dev.hadoop.apache.org%3E > > > > > > On Wed, Jun 5, 2019 at 3:44 PM Xiaoqiao He wrote: > > > > > Thanks Brahma for starting the thread. > > > +1 for merging. > > > > > > He Xiaoqiao > > > > > > On Tue, Jun 4, 2019 at 1:53 PM Chittaranjan Hota > > > > > > wrote: > > > > > >> Thanks Brahma initiating this. > > >> +1(non-binding) for merge. > > >> > > >> @Uber we have almost all changes specially rbf security in productio= n > > >> for a while now without issues. > > >> > > >> On Mon, Jun 3, 2019 at 12:56 PM Giovanni Matteo Fumarola < > > >> giovanni.fumarola@gmail.com> wrote: > > >> > > >> > +1 on merging. > > >> > > > >> > Thanks Brahma for starting the thread. > > >> > > > >> > On Mon, Jun 3, 2019 at 10:00 AM I=C3=B1igo Goiri > > wrote: > > >> > > > >> > > Thank you Brahma for pushing this. > > >> > > > > >> > > As you mentioned, we have already taken most of the changes into > > >> > > production. > > >> > > I want to highlight that the main contribution is the addition o= f > > >> > security. > > >> > > We have been able to test this at a smaller scale (~500 servers > > >> > > and 4 > > >> > > subclusters) and the performance is great with our current > > >> > > ZooKeeper deployment. > > >> > > I would also like to highlight that all the changes are > > >> > > constrained to hadoop-hdfs-rbf and there is no differences in > > commons or HDFS. > > >> > > > > >> > > +1 on merging > > >> > > > > >> > > Inigo > > >> > > > > >> > > On Sun, Jun 2, 2019 at 10:19 PM Akira Ajisaka > > >> > > > > >> > wrote: > > >> > > > > >> > > > Thanks Brahma for starting the discussion. > > >> > > > I'm +1 for merging this. > > >> > > > > > >> > > > FYI: In Yahoo! JAPAN, deployed all these changes in 20 nodes > > >> > > > cluster with 2 routers (not in production) and running several > > tests. > > >> > > > > > >> > > > Regards, > > >> > > > Akira > > >> > > > > > >> > > > On Sun, Jun 2, 2019 at 12:40 PM Brahma Reddy Battula < > > >> > brahma@apache.org> > > >> > > > wrote: > > >> > > > > > > >> > > > > Dear Hadoop Developers > > >> > > > > > > >> > > > > I would like to propose RBF Branch (HDFS-13891) merge into > > trunk. > > >> We > > >> > > have > > >> > > > > been working on this feature from last several months. > > >> > > > > This feature work received the contributions from different > > >> > companies. > > >> > > > All > > >> > > > > of the feature development happened smoothly and > > >> > > > > collaboratively > > >> in > > >> > > > JIRAs. > > >> > > > > > > >> > > > > Kindly do take a look at the branch and raise issues/concern= s > > >> > > > > that > > >> > need > > >> > > > to > > >> > > > > be addressed before the merge. > > >> > > > > > > >> > > > > *Highlights of HDFS-13891 Branch:* > > >> > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D > > >> > > > > > > >> > > > > Adding Security to RBF(1) > > >> > > > > Adding Missing Client API's(2) Improvements/Bug Fixing > > >> > > > > Critical - HDFS-13637, HDFS-13834 > > >> > > > > > > >> > > > > *Commits:* > > >> > > > > =3D=3D=3D=3D=3D=3D=3D=3D > > >> > > > > > > >> > > > > No of JIRAs Resolved: 72 > > >> > > > > > > >> > > > > All this commits are in RBF Module. No changes in hdfs/commo= n. > > >> > > > > > > >> > > > > *Tested Cluster:* > > >> > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > >> > > > > > > >> > > > > Most of these changes verified at Uber,Microsoft,Huawei and > > >> > > > > some > > >> > other > > >> > > > > compaines. > > >> > > > > > > >> > > > > *Uber*: Most changes are running in production @Uber > > >> > > > > including the > > >> > > > critical > > >> > > > > security changes, HDFS Clusters are 4000+ nodes with 8 HDFS > > >> Routers. > > >> > > > > Zookeeper as a state store to hold delegation tokens were > > >> > > > > also > > >> stress > > >> > > > > tested to hold more than 2 Million tokens. --CR Hota > > >> > > > > > > >> > > > > *MicroSoft*: Most of these changes are currently running in > > >> > production > > >> > > at > > >> > > > > Microsoft.The security has also been tested in a 500 server > > >> cluster > > >> > > with > > >> > > > 4 > > >> > > > > subclsuters. --Inigo Goiri > > >> > > > > > > >> > > > > *Huawei* : Deployed all this changes in 20 node cluster with > > >> > > > > 3 routers.Planning deploy 10K production cluster. > > >> > > > > > > >> > > > > *Contributors:* > > >> > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > >> > > > > > > >> > > > > Many thanks to Akira Ajisaka,Mohammad Arshad,Takanobu > > >> Asanuma,Shubham > > >> > > > > Dewan,CR Hota,Fei Hui,Inigo Goiri,Dibyendu Karmakar,Fengna > > >> > > > > Li,Gang Li,Surendra Singh Lihore,Ranith Sardar,Ayush > > >> > > > > Saxena,He > > >> > > Xiaoqiao,Sherwood > > >> > > > > Zheng,Daryn Sharp,VinayaKumar B,Anu Engineer for invloving > > >> > discussions > > >> > > > and > > >> > > > > contributing to this. > > >> > > > > > > >> > > > > *Future Tasks:* > > >> > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > >> > > > > > > >> > > > > will cleanup the jira's under this umbrella and contiue to > > work. > > >> > > > > > > >> > > > > Reference: > > >> > > > > 1) https://issues.apache.org/jira/browse/HDFS-13532 > > >> > > > > 2) https://issues.apache.org/jira/browse/HDFS-13655 > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > --Brahma Reddy Battula > > >> > > > > > >> > > > > > >> --------------------------------------------------------------------= - > > >> > > > To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org > > >> > > > For additional commands, e-mail: > > >> > > > hdfs-dev-help@hadoop.apache.org > > >> > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > >> -- > > >> Regards, > > >> Chitta > > > > > > > > >> > > > > -- > > > > > > > > --Brahma Reddy Battula > > -- > > > > > > > > --Brahma Reddy Battula > > -- > > > > > > > > --Brahma Reddy Battula > > > --=20 --Brahma Reddy Battula --=20 --Brahma Reddy Battula --000000000000f4c000058b75e1e8--