From hadoop-migrations-return-9-archive-asf-public=cust-asf.ponee.io@infra.apache.org Thu Apr 30 20:50:09 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 21B98180630 for ; Thu, 30 Apr 2020 22:50:09 +0200 (CEST) Received: (qmail 34885 invoked by uid 500); 30 Apr 2020 20:50:08 -0000 Mailing-List: contact hadoop-migrations-help@infra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-migrations@infra.apache.org Delivered-To: mailing list hadoop-migrations@infra.apache.org Received: (qmail 34868 invoked by uid 99); 30 Apr 2020 20:50:08 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 30 Apr 2020 20:50:08 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id A21EBC0A2E for ; Thu, 30 Apr 2020 20:50:07 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id EdlsUOtkPitJ for ; Thu, 30 Apr 2020 20:50:05 +0000 (UTC) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::241; helo=mail-oi1-x241.google.com; envelope-from=ipv6guru@gmail.com; receiver= Received: from mail-oi1-x241.google.com (mail-oi1-x241.google.com [IPv6:2607:f8b0:4864:20::241]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTPS id 15D847D3FB for ; Thu, 30 Apr 2020 20:50:05 +0000 (UTC) Received: by mail-oi1-x241.google.com with SMTP id x10so924709oie.1 for ; Thu, 30 Apr 2020 13:50:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=2DrGoI/lOndDCR61j2TEqx0QRBUNOkjpltc4+P6IKgU=; b=hA77w6SPMRaBqO9gI05gM2F1gV8oFdEZsSUQFr1/6U1UPfevgU1It2+fBhh60gd2Wo K5hsex7lx3aGZJJ5Go8Qy+U9MTAq7WjRDB0qUR6M04mz46H57RfMjvyJXcw9qBU1PCrU 1qpvR81Ve8ZMBWefI3twVHxFJsEhNsA2vH5FKG03GjFPjWuaONOjxddCnaT+82dg4pPW aeLfzS/k46Yzzpm5SC0oCxd5KMp/41gvnJtCE8zF3BZxFuq6J/xteSKIVGKdAB6rKg/6 3E8oJvcamz0MMRSvHDhdHZ9Sf9VBLYZrA3g9VXVtn3jZg86hqolteoKFVa2xid88Kgo9 29SQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=2DrGoI/lOndDCR61j2TEqx0QRBUNOkjpltc4+P6IKgU=; b=V3PWPQ2L7482VxcPU5LTDAYMkpMuwm6ngQ6926eLzSSACGHM9Q0HoZw2mGxNeoT2+k //COjQZaiARIPXXqLjRxEPkzB/xi6hYHqwEHhaCWc1jnz1kQ1S4Cy5RnbvlhDgrhf1ss RS1vd7xziTfgckWVusfocKZBqjpLgdfFGPLATd9WazNPt4wCvYvlqOQE+s35/NFW5MKW m/Tp9n8KUyR5sFixN+ZU77/ieka9Yu4bY10+Hx7pX/zqQRWwvOORxZEOQF5J2tTExtY6 QRYVH1/PqVIvBYvVxqAf8WIz1jKev4wV3NrFAS6eyGTxEHEifajtPpapCCE01NDLqlvw Dm8w== X-Gm-Message-State: AGi0Puaj/1gblM2Iwy3oKW+l9d6rjLppR+QDhNaLyujLRiROi+ygVXWF 5uxnsvKHgroy3MVs4D9N9J+eCwZXfMf3PgpyS2tmSB/+ X-Google-Smtp-Source: APiQypLn4WyL7pRNc8lBvT4b1IdCWlGodtFlpcA91lBhREkOQe60qQZAREkBc7VeZI18AYarrF4lB/NLm1BZOfhBTwM= X-Received: by 2002:a05:6808:3d5:: with SMTP id o21mr807405oie.40.1588279797688; Thu, 30 Apr 2020 13:49:57 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Gavin McDonald Date: Thu, 30 Apr 2020 22:49:46 +0200 Message-ID: Subject: Re: [DISCUSS] Migrating HBase to new CI Master (was Re: Migration of Hadoop labelled nodes to new dedicated Master) To: hadoop-migrations@infra.apache.org Content-Type: multipart/alternative; boundary="000000000000252d8f05a48835c6" --000000000000252d8f05a48835c6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Apr 30, 2020 at 10:10 PM Sean Busbey wrote: > Can we please bulk grant all HBase committers setup permissions as a > starting point? Presuming there are documented steps the PMC can do > incremental updates as we vote in new committers. > Yes, perhaps you missed my earlier mail where I said I have fixed it for all of HBase. HTH Gav... > > On Thu, Apr 30, 2020 at 1:36 AM =E5=BC=A0=E9=93=8E(Duo Zhang) > wrote: > > > Please also give me the permission for helping setup the hbase related > > jobs. > > > > Thanks. > > > > Nick Dimiduk =E4=BA=8E2020=E5=B9=B44=E6=9C=8830= =E6=97=A5=E5=91=A8=E5=9B=9B =E4=B8=8A=E5=8D=881:49=E5=86=99=E9=81=93=EF=BC= =9A > > > > > Hi Gavin, > > > > > > I'd like to get started by copying over our nightly job to run on the > new > > > master. I've accessed ci-hadoop.apache.org and find that I'm able to > log > > > in > > > with my Apache ID. However, I don't seem to have permissions to creat= e > a > > > new job. Can you please grant me appropriate karma? > > > > > > Next question is with migrating itself. Is it possible for you to > restore > > > the current job XML into a job on new master? This preferable vs. us > > > attempting to manually define the job via the UI. The job in question > is > > > https://builds.apache.org/job/HBase%20Nightly/ > > > > > > Thanks a lot, > > > Nick > > > > > > On Wed, Apr 29, 2020 at 6:06 AM Gavin McDonald > > > wrote: > > > > > > > Hi All, > > > > > > > > Following on from the below email I sent *11 DAYS ago now*, so far = we > > > have > > > > had *one reply *from mahout cc:d to me (thank you Trevor) , and had > > *ONE > > > > PERSON* sign up to the new hadoop-migrations@infra.apache.org - > > > > that is out of a total of *OVER 7000* people signed up to the 13 > > mailing > > > > lists emailed. > > > > > > > > To recap what I asked for:- > > > > > > > > "...What I would like from each community, is to decide who is goin= g > to > > > > help with their project in performing these migrations - ideally 2 > or 3 > > > > folks who use the current builds.a.o regularly. Those folks should > then > > > > subscribe to the new dedicated hadoop-migrations@infra.apache.org > > > mailing > > > > lists as soon as possible so we can get started..." > > > > > > > > This will be last email I sent to your dev list directly. I am now > > > building > > > > a new Jenkins Master, and as soon as it is ready I will start to > > migrate > > > > the Jenkins Nodes/Agents over to the new system. > > > > And; when I am done, the existing builds.apache.org *WILL BE TURNED > > > OFF*. > > > > > > > > I am now going to continue all conversations on the > > > > hadoop-migrations@infra.apache.org list *only.* > > > > > > > > Thanks > > > > > > > > Gavin McDonald (ASF Infra) > > > > > > > > > > > > On Sat, Apr 18, 2020 at 4:21 PM Gavin McDonald > > > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > A couple of months ago, I wrote to a few project private lists > > > mentioning > > > > > the need to migrate Hadoop labelled nodes (H0-H21) over to a new > > > > dedicated > > > > > Jenkins Master [1] (a Cloudbees Client Master.). > > > > > > > > > > I'd like to revisit this now that I have more time to dedicate to > > > getting > > > > > this done. However, keeping track across multiple mailing lists, > > > > > separate conversations that spring up in various places is > cumbersome > > > and > > > > > not realistic. To that end, I have created a new specific mailing > > list > > > > > dedicated to the migrations of these nodes, and the projects that > use > > > > them, > > > > > over to the new system. > > > > > > > > > > The mailing list 'hadoop-migrations@infra.apache.org' is up and > > > running > > > > > now (and this will be the first post to it). Previous discussions > > were > > > on > > > > > the private PMC lists, (there was some debate about that but I > wanted > > > the > > > > > PMCs initially to be aware of the change,) this new list is publi= c > > and > > > > > archived. > > > > > > > > > > This email is BCC'd to 13 projects dev lists [2] determined by th= e > > > > https:/ > > > > > hadoop.apache.org list of Related projects, minus Cassandra whom > > > already > > > > > have their own dedicated client master [3] and I added Yetus as I > > think > > > > > they cross collaborate with many Hadoop based projects. If anyone > > > thinks > > > > a > > > > > project is missing, or should not be on the list, let me know. > > > > > > > > > > What I would like from each community, is to decide who is going = to > > > help > > > > > with their project in performing these migrations - ideally 2 or = 3 > > > folks > > > > > who use the current builds.a.o regularly. Those folks should then > > > > subscribe > > > > > to the new dedicated hadoop-migrations@infra.apache.org mailing > > lists > > > as > > > > > soon as possible so we can get started. > > > > > > > > > > About the current setup - and I hope this answers previously aske= d > > > > > questions on private lists - the new dedicated master is a > Cloudbees > > > > Client > > > > > Master 2.204.3.7-rolling. It is not the same setup as the current > > > Jenkins > > > > > master on builds.a.o - it is not intended to be. It is more or > less a > > > > > 'clean install' in that I have not installed over 500 plugins as = is > > the > > > > > case on builds.a.o , I would rather we install plugins as we find > we > > > need > > > > > them. So yes, there may be some features missing - the point of > > having > > > > > people sign up to the new list is to find out what those are, get > > them > > > > > installed, and get your builds to at least the same state they ar= e > in > > > > > currently. > > > > > > > > > > We have 2 nodes on there currently for testing, as things progres= s > we > > > can > > > > > transfer over a couple more, projects can start to migrate their > jobs > > > > over > > > > > at any time they are happy , until done. We also need to test aut= h > - > > > the > > > > > master; and its nodes will be restricted to just Hadoop + Related > > > > projects > > > > > (which is important this list of related projects is correct). No > > > longer > > > > > will other projects be able to hop on to Hadoop nodes, and no > longer > > > will > > > > > Hadoop related projects be able to hop onto other folks nodes. Th= is > > is > > > a > > > > > good thing, and may encourage some providers to donate a few more > VMs > > > for > > > > > dedicated use. > > > > > > > > > > For now then, decide who will help with this process, and sign up > to > > > the > > > > > new mailing list, and lets get started! > > > > > > > > > > Note I am NOT subscribed to any of your dev lists, so replies > please > > cc > > > > > the new list. and I will await your presence there to get started= . > > > > > > > > > > Thanks all. > > > > > > > > > > Gavin McDonald (ASF Infra) > > > > > > > > > > [1] - https://ci-hadoop.apache.org > > > > > [2] - > > > > > > > > > > > > > > > hadoop,chukwa,avro,ambari,hbase,hive,mahout,pig,spark,submarine,tez,zooke= eper,yetus > > > > > [3] - https://ci-cassandra.apache.org > > > > > > > > > > > > > > > --=20 Gav... --000000000000252d8f05a48835c6--