Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 83DBE200C72 for ; Fri, 12 May 2017 21:58:37 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 82493160BB8; Fri, 12 May 2017 19:58:37 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id A072C160BA8 for ; Fri, 12 May 2017 21:58:36 +0200 (CEST) Received: (qmail 51828 invoked by uid 500); 12 May 2017 19:58:33 -0000 Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list common-dev@hadoop.apache.org Received: (qmail 51600 invoked by uid 99); 12 May 2017 19:58:32 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 12 May 2017 19:58:32 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 2F73A1804F7; Fri, 12 May 2017 19:58:32 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.381 X-Spam-Level: ** X-Spam-Status: No, score=2.381 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, HTML_OBFUSCATE_05_10=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id nml6JmKQgpXa; Fri, 12 May 2017 19:58:29 +0000 (UTC) Received: from mail-qt0-f173.google.com (mail-qt0-f173.google.com [209.85.216.173]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id AC0D55F36B; Fri, 12 May 2017 19:58:28 +0000 (UTC) Received: by mail-qt0-f173.google.com with SMTP id t26so49997413qtg.0; Fri, 12 May 2017 12:58:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=1nivwJlQp6QgXqCZUWIfkPxnu4FQsXgeg0xUarB/Nus=; b=V+J4/svLrmSWlunoZGJlmv+uW+CUQopKt0g6VEIA6F78jy07XqsPKnNFclX8jhFnPQ 4jwjKj5JCyA863zyJc0oCgd+B12BZysMS2RuR4pHqH3QILlL/J662AUDBvIwdsuC5b6m 1zmplH7W4mjbAbs+KR4lYVn0ErZGsjUQe9fr0E+KHcs4R+c7oWLpSut3+iANideqA2hf /3xFZQyjcYvAhyeHxsTHAFvoDqJwFFd26Y+5gl8z+WQX0Ufi3w4k2HeulzDGWSMLBdVy pR0Yb8Z8+oSfuB/icdjjBqRV/0cEdfmskT6h8tYqXZspebwN+DkDBuNu/tPhWwB7mCnp pWrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=1nivwJlQp6QgXqCZUWIfkPxnu4FQsXgeg0xUarB/Nus=; b=uEbiFn5KMsECm4+zzaKgx0Q3+A7Js/eCm8D3K7p0aZohx7lRH/xaubPggd5qV267TL UE791+9W8ynoMfNn+z9fkMgGIkS2stFqQrGe3ZmgMcQfrRAA9C1Ws6C7GYPfoaX5/yXM 25zyEX83jwGj/qYCKzzk/Tk4+TXODtaIoto9cqz5JpN9LBaoQO/R1K+2gVsmf93toDgJ UA0flVg7MX1Yc4vz2Re0ZdiZIORFkh5Zy8PJdf/LQ98lJ+otNKMYqPT9KTHbp8fFOG2e pmJQBaGMqGRyn9IhTUipRNee+KXQNt/+MJ3C761nuGtUe/yYaC1z4UTts2wQqFNMmsQ/ icSw== X-Gm-Message-State: AODbwcCiW0tzRExgVfTdbqz7qww3dfA4In2lpe5yiLCq4IBR/UihB3Kk i3AIqHsc8Q4qWpDRMtqTgO3aFKGVXA== X-Received: by 10.237.44.129 with SMTP id g1mr5775810qtd.212.1494619102627; Fri, 12 May 2017 12:58:22 -0700 (PDT) MIME-Version: 1.0 Received: by 10.12.186.165 with HTTP; Fri, 12 May 2017 12:58:21 -0700 (PDT) In-Reply-To: References: <8AD4EE147886274A8B495D6AF407DF698F25D8BE@blreml510-mbx.china.huawei.com> From: Konstantin Shvachko Date: Fri, 12 May 2017 12:58:21 -0700 Message-ID: Subject: Re: About 2.7.4 Release To: Akira Ajisaka Cc: Hadoop Common , Hdfs-dev , "mapreduce-dev@hadoop.apache.org" , "yarn-dev@hadoop.apache.org" Content-Type: multipart/alternative; boundary="94eb2c12560aafcbfb054f592010" archived-at: Fri, 12 May 2017 19:58:37 -0000 --94eb2c12560aafcbfb054f592010 Content-Type: text/plain; charset="UTF-8" Latest update on the links and filters. Here is the correct link for the filter: https://issues.apache.org/jira/secure/IssueNavigator.jspa?requestId=12340814 Also updated: https://s.apache.org/Dzg4 Had to do some Jira debugging. Sorry for confusion. Thanks, --Konstantin On Wed, May 10, 2017 at 2:30 PM, Konstantin Shvachko wrote: > Hey Akira, > > I didn't have private filters. Most probably Jira caches something. > Your filter is in the right direction, but for some reason it lists only > 22 issues, while mine has 29. > It misses e.g. YARN-5543 > . > > Anyways, I created a Jira filter now "Hadoop 2.7.4 release blockers", > shared it with "everybody", and updated my link to point to that filter. So > you can use any of the three methods below to get the correct list: > 1. Go to https://s.apache.org/Dzg4 > 2. Go to the filter via > https://issues.apache.org/jira/issues?filter=12340814 > or by finding "Hadoop 2.7.4 release blockers" filter in the jira > 3. On Advanced issues search page paste this: > project in (HDFS, HADOOP, YARN, MAPREDUCE) AND labels = release-blocker > AND "Target Version/s" = 2.7.4 > > Hope this solves the confusion for which issues are included. > Please LMK if it doesn't, as it is important. > > Thanks, > --Konstantin > > On Tue, May 9, 2017 at 9:58 AM, Akira Ajisaka wrote: > >> Hi Konstantin, >> >> Thank you for volunteering as release manager! >> >> > Actually the original link works fine: https://s.apache.org/Dzg4 >> I couldn't see the link. Maybe is it private filter? >> >> Here is a link I generated: https://s.apache.org/ehKy >> This filter includes resolved issue and excludes fixversion == 2.7.4 >> >> Thanks and Regards, >> Akira >> >> On 2017/05/08 19:20, Konstantin Shvachko wrote: >> >>> Hi Brahma Reddy Battula, >>> >>> Actually the original link works fine: https://s.apache.org/Dzg4 >>> Your link excludes closed and resolved issues, which needs backporting, >>> and >>> which we cannot reopen, as discussed in this thread earlier. >>> >>> Looked through the issues you proposed: >>> >>> HDFS-9311 >>> Seems like a new feature. It helps failover to standby node when primary >>> is >>> under heavy load, but it introduces new APIs, addresses, config >>> parameters. >>> And needs at least one follow up jira. >>> Looks like a backward compatible change, though. >>> Did you have a chance to run it in production? >>> >>> +1 on >>> HDFS-10987 >>> HDFS-9902 >>> HDFS-8312 >>> HADOOP-14100 >>> >>> Added them to 2.7.4 release. You should see them via the above link now. >>> Would be good if you could attach backport patches for some of them? >>> >>> Appreciate your help, >>> --Konstantin >>> >>> On Mon, May 8, 2017 at 8:39 AM, Brahma Reddy Battula < >>> brahmareddy.battula@huawei.com> wrote: >>> >>> >>>> Looks following link is not correct.. >>>> >>>> https://s.apache.org/Dzg4 >>>> >>>> It should be like following..? >>>> >>>> https://s.apache.org/wi3U >>>> >>>> >>>> Apart from Konstantin mentioned,Following also good to go..? let me know >>>> your thoughts on this. >>>> >>>> For Large Cluster: >>>> ============= >>>> >>>> https://issues.apache.org/jira/browse/HDFS-9311===Life Line Protocol >>>> https://issues.apache.org/jira/browse/HDFS-10987===Deecommission >>>> Expensive when lot's of blocks are present >>>> >>>> https://issues.apache.org/jira/browse/HDFS-9902=== >>>> "dfs.datanode.du.reserved" per Storage Type >>>> >>>> For Security: >>>> ========= >>>> https://issues.apache.org/jira/browse/HDFS-8312===Trash does not >>>> descent >>>> into child directories to check for permission >>>> https://issues.apache.org/jira/browse/HADOOP-14100===Upgrade Jsch jar >>>> to >>>> latest version to fix vulnerability in old versions >>>> >>>> >>>> >>>> Regards >>>> Brahma Reddy Battula >>>> >>>> -----Original Message----- >>>> From: Erik Krogen [mailto:ekrogen@linkedin.com.INVALID] >>>> Sent: 06 May 2017 02:40 >>>> To: Konstantin Shvachko >>>> Cc: Zhe Zhang; Hadoop Common; Hdfs-dev; mapreduce-dev@hadoop.apache.org >>>> ; >>>> yarn-dev@hadoop.apache.org >>>> Subject: Re: About 2.7.4 Release >>>> >>>> List LGTM Konstantin! >>>> >>>> Let's say that we will only create a new tracking JIRA for patches which >>>> do not backport cleanly, to avoid having too many lying around. >>>> Otherwise >>>> we can directly attach to old ticket. If a clean backport does happen to >>>> break a test the nightly build will help us catch it. >>>> >>>> Erik >>>> >>>> On Thu, May 4, 2017 at 7:21 PM, Konstantin Shvachko < >>>> shv.hadoop@gmail.com> >>>> wrote: >>>> >>>> Great Zhe. Let's monitor the build. >>>>> >>>>> I marked all jiras I knew of for inclusion into 2.7.4 as I described >>>>> before. >>>>> Target Version/s: 2.7.4 >>>>> Label: release-blocker >>>>> >>>>> Here is the link to the list: https://s.apache.org/Dzg4 Please let me >>>>> know if I missed anything. >>>>> And feel free to pick up any. Most of backports are pretty >>>>> straightforward, but not all. >>>>> >>>>> We can create tracking jiras for backporting if you need to run >>>>> Jenkins on the patch (and since Allen does not allow reopening them). >>>>> But I think the final patch should be attached to the original jira. >>>>> Otherwise history will be hard to follow. >>>>> >>>>> Thanks, >>>>> --Konstantin >>>>> >>>>> On Wed, May 3, 2017 at 4:53 PM, Zhe Zhang wrote: >>>>> >>>>> Thanks for volunteering as RM Konstantin! The plan LGTM. >>>>>> >>>>>> I've created a nightly Jenkins job for branch-2.7 (unit tests): >>>>>> https://builds.apache.org/job/Hadoop-branch2.7-nightly/ >>>>>> >>>>>> On Wed, May 3, 2017 at 12:42 AM Konstantin Shvachko < >>>>>> >>>>> shv.hadoop@gmail.com> >>>>> >>>>>> wrote: >>>>>> >>>>>> Hey guys, >>>>>>> >>>>>>> I and a few of my colleagues would like to help here and move 2.7.4 >>>>>>> release forward. A few points in this regard. >>>>>>> >>>>>>> 1. Reading through this thread since March 1 I see that Vinod >>>>>>> hinted on managing the release. Vinod, if you still want the job / >>>>>>> have bandwidth will be happy to work with you. >>>>>>> Otherwise I am glad to volunteer as the release manager. >>>>>>> >>>>>>> 2. In addition to current blockers and criticals, I would like to >>>>>>> >>>>>> propose >>>>> >>>>>> a >>>>>>> few issues to be included in the release, see the list below. Those >>>>>>> are mostly bug fixes and optimizations, which we already have in >>>>>>> our >>>>>>> >>>>>> internal >>>>> >>>>>> branch and run in production. Plus one minor feature "node >>>>>>> labeling", which we found very handy, when you have heterogeneous >>>>>>> environments and mixed workloads, like MR and Spark. >>>>>>> >>>>>>> 3. For marking issues for the release I propose to >>>>>>> - set the target version to 2.7.4, and >>>>>>> - add a new label "release-blocker" >>>>>>> That way we will know issues targeted for the release without >>>>>>> reopening them for backports. >>>>>>> >>>>>>> 4. I see quite a few people are interested in the release. With all >>>>>>> the help I think we can target to release by the end of May. >>>>>>> >>>>>>> Other things include fixing CHANGES.txt and fixing Jenkins build >>>>>>> for >>>>>>> >>>>>> 2.7.4 >>>>> >>>>>> branch. >>>>>>> >>>>>>> Thanks, >>>>>>> --Konstantin >>>>>>> >>>>>>> ========== List of issue for 2.7.4 =========== >>>>>>> ------ Backports >>>>>>> HADOOP-12975 . >>>>>>> Add >>>>>>> >>>>>> du >>>>> >>>>>> jitters >>>>>>> HDFS-9710 . IBR >>>>>>> >>>>>> batching >>>>> >>>>>> HDFS-10715 . NPE >>>>>>> when applying AvailableSpaceBlockPlacementPolicy >>>>>>> HDFS-2538 . fsck >>>>>>> >>>>>> removal >>>>> >>>>>> of dot printing >>>>>>> HDFS-8131 . >>>>>>> space-balanced >>>>>>> policy for balancer >>>>>>> HDFS-8549 . abort >>>>>>> balancer if upgrade in progress >>>>>>> HDFS-9412 . skip >>>>>>> small blocks in getBlocks >>>>>>> >>>>>>> YARN-1471 . SLS >>>>>>> simulator >>>>>>> YARN-4302 . SLS >>>>>>> YARN-4367 . SLS >>>>>>> YARN-4612 . SLS >>>>>>> >>>>>>> ----- Node labeling >>>>>>> MAPREDUCE-6304 >>>>>>> >>>>>>> YARN-2943 >>>>>>> YARN-4109 >>>>>>> YARN-4140 >>>>>>> YARN-4250 >>>>>>> YARN-4925 >>>>>>> >>>>>>> -- >>>>>> Zhe Zhang >>>>>> Apache Hadoop Committer >>>>>> http://zhe-thoughts.github.io/about/ | @oldcap >>>>>> >>>>>> >>>>> >>>> >>> > --94eb2c12560aafcbfb054f592010--