From dev-return-2021-archive-asf-public=cust-asf.ponee.io@mxnet.incubator.apache.org Thu Jan 25 15:51:39 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 3829C180651 for ; Thu, 25 Jan 2018 15:51:39 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 280D8160C3D; Thu, 25 Jan 2018 14:51:39 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id EAE61160C13 for ; Thu, 25 Jan 2018 15:51:37 +0100 (CET) Received: (qmail 21049 invoked by uid 500); 25 Jan 2018 14:51:37 -0000 Mailing-List: contact dev-help@mxnet.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@mxnet.incubator.apache.org Delivered-To: mailing list dev@mxnet.incubator.apache.org Received: (qmail 21031 invoked by uid 99); 25 Jan 2018 14:51:36 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Jan 2018 14:51:36 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id E0FAD1A0CBD for ; Thu, 25 Jan 2018 14:51:35 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.379 X-Spam-Level: ** X-Spam-Status: No, score=2.379 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, KAM_NUMSUBJECT=0.5, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=googlemail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id BK2wyXHFIbxF for ; Thu, 25 Jan 2018 14:51:30 +0000 (UTC) Received: from mail-lf0-f44.google.com (mail-lf0-f44.google.com [209.85.215.44]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id C07185F23E for ; Thu, 25 Jan 2018 14:51:29 +0000 (UTC) Received: by mail-lf0-f44.google.com with SMTP id x196so10121113lfd.12 for ; Thu, 25 Jan 2018 06:51:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=+t7E8dWGprSkpzEXgG0lwu6cLIZE3qtdg4z3yrzO4E0=; b=gChUiSl/0YK105xe+anXXlhJo0elpDrKMDiKLo5xdcu/xkKUNGooxrZKeCrDUii9IU K1B7uVAnvMKTWSrcZsIvAlGaBu3Yw8h7Uv2UOleg21Nhxid7ZyBIxFwQWf0ISB5Z9eAR mmqGI2K8Ps/Ux95Gmb2u0a5NWjpNL+0lqDVPhlwmuqBvLl4cJf1TPcIJtftLsJjxoDeW uCX5y/YZ7vlF7VWJTAsZzUcEBVoDfMS0VbZ1ouTUeXX0NroE2tTm2CPD/A/EEdpIyIIM 1aYybRO5hBpUTQ2b/imKz6FpjOwu1U7sodihO5xRqUEji1CHbdHu0WLL+V6dGnb8cYzc CIjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=+t7E8dWGprSkpzEXgG0lwu6cLIZE3qtdg4z3yrzO4E0=; b=unbITNqREw1h99VRDsh4bYcEM0vMyYa/vprjV8ReKjjFSd1PpfsZZqhus4J5EFoydP oJ0YCLKzHfRFrgyrzQLWoOFnxjNnz+sBb3rmiWHYenS7MEy9MelgdX+yg9DxL8ISasDs 73eFsJ9aOH+r2lcpvCYGovhl2zIlZLwnQop6WcqjFE+W1Odtwv8xTU/o9t+XsNFvlUaL 293G57zSeSvGiG0/ETwnIj9MUGcsEwWPaYr+onLooov6oXsRlDUiUjeWECuSleu5mdlL Q+MhFuqzH0XswoSuyJrcL2fHHtbmPrrLs/slhPnwoEITjeb/BnwZ4jZ610WdsAqD6yNl 2JtA== X-Gm-Message-State: AKwxytcLF6xksBLHKZCONPcsovfpMcbyfFJtcNGa/WT4Pf+Dzy848Vhl f7TOYRUotVElDLD05a8MYvs5R2ZGBBjhyqvk2LM= X-Google-Smtp-Source: AH8x226onlger4K7DA/hF6ptKrbNlP7JoazB8+/iW/Vj+jtG6OE3vgpzX31pRGTQlHrqjEJWuwlYttK2ToLK4zzb9i4= X-Received: by 10.46.77.2 with SMTP id a2mr5985310ljb.69.1516891888385; Thu, 25 Jan 2018 06:51:28 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.169.206 with HTTP; Thu, 25 Jan 2018 06:51:27 -0800 (PST) Received: by 10.25.169.206 with HTTP; Thu, 25 Jan 2018 06:51:27 -0800 (PST) In-Reply-To: References: <00FBDC6D-1917-4CFB-B7F4-6C2955C9DA43@amazon.com> From: Marco de Abreu Date: Thu, 25 Jan 2018 06:51:27 -0800 Message-ID: Subject: Re: Release plan - MXNET 1.0.1 To: dev@mxnet.incubator.apache.org Content-Type: multipart/alternative; boundary="94eb2c1ab8ca2b825105639aeaad" --94eb2c1ab8ca2b825105639aeaad Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable 1. I'm afraid we got quite a lot of APIs, especially considering the various Interface languages we support. It would be quite hard and error prone to verify this. 2. I'm not looking into all PRs, but I can't recall that much attention is being paid to this fact. I only remember there was a PR from Pedro regarding changing the pointer type, but I can't recall the outcome. 3. I've spoken to quite a few people recently and I noticed that not everyone is subscribed to this email lists. Unfortunately, this approach would also be error prone. 4. I'm not aware of many tests in that area. There is some integration for the caffee converter and a very small nightly job that trains on the latest version and runs inference on the previous version, but I dont know which operators are covered. In general I would say that we don't have enough integration test coverage, but I'd be happy to be told otherwise. I'd say they are not sufficient for us to trust. -Marco Am 25.01.2018 6:42 vorm. schrieb "kellen sunderland" < kellen.sunderland@gmail.com>: By my count we have four mechanisms in place to verify no breaking API changes have taken place: 1) Have the important public api's changed? This can be verified in by inspecting git history of important interfaces. 2) Has anyone indicated in a PR that their change was a breaking change (reviewers should be aware if so). 3) Email the dev list and ask specifically if there have been breaking changes done, it seems scala is the only breaking change that I've seen. 4) Integration tests from several languages that run both per PR and nightly. If breaking changes happen ideally these should catch them provided we have enough coverage. If they've been updated to match a breaking change this should also be apparent to reviewers. Of course even with these mechanisms in place breaking changes can slip through. I think this is going to be tough to prevent, especially with the degree of subrepos we have. Would you disagree that these mechanisms exist, or are you of the opinion that they just aren't sufficient for us to trust? On Thu, Jan 25, 2018 at 3:29 PM, Marco de Abreu < marco.g.abreu@googlemail.com> wrote: > Hi Kellen, > > The reason I proposed a minor release is the matter of the fact that we d= o > not know if we have no breaking changes. We currently have no mechanism i= n > place to validate this fact and while you are right that this will probably > result in a slower adoption rate, I'm more afraid about users just > upgrading with the expectancy that no API changes were introduced, while it > may actually has happened. > > As long as we got no mechanism in place to validate the preconditions to > make this a patch release, we shouldn't act like we guarantee it. > > -Marco > > Am 25.01.2018 2:01 vorm. schrieb "kellen sunderland" < > kellen.sunderland@gmail.com>: > > > -.5 (non-binding) to releasing as a minor release. If we don't have an= y > > breaking API changes, and we haven't added any major features I would > tend > > to release this as a patch release. The reason being that organization= s > > and users will know that they can apply this release without making major > > changes to their dependencies. It also helps set expectations around the > > degree of regression testing you'd expect to do on a release (typically > > patch releases would require less testing). For that reason if we > release > > as a patch release I think we could expect better adoption rates in the > > community and within large tech orgs. If we release as a minor release > we > > should expect that many customers may take a long time to update, and a= s > a > > community we will be forced to provide support for bugs which have > already > > been fixed. > > > > +1 (non-binding) In terms of branching I'd agree that we should apply > most > > of the fixes to the previous release branch and build from there. Happ= y > to > > help with this if needed. > > > > On Thu, Jan 25, 2018 at 6:19 AM, Nan Zhu wrote= : > > > > > +1 and suggest consolidating all maintenance releases under the same > > > major.minor version into a single branch > > > > > > On Wed, Jan 24, 2018 at 9:06 PM, Meghna Baijal < > > meghnabaijal2017@gmail.com > > > > > > > wrote: > > > > > > > I agree. If the release candidate is being cut from the master > branch, > > it > > > > should be considered a minor release. > > > > > > > > Anyway the effort involved in the release process is exactly the same > > in > > > > either case. > > > > > > > > Thanks, > > > > Meghna > > > > > > > > On Jan 24, 2018 8:56 PM, "Marco de Abreu" < > > marco.g.abreu@googlemail.com> > > > > wrote: > > > > > > > > > Are there any particular reasons why we are classifying this > release > > as > > > > > patch instead of minor release? As far as I know, we don't have any > > > tests > > > > > in place to determine API changes and thus can't guarantee that > this > > is > > > > an > > > > > actual patch release. Considering the fact that PRs have been > merged > > > > > without having semantic versioning in place, this could be quite > > risky. > > > > > > > > > > Instead, I'd rather propose to make a minor release 1.1 instead o= f > > > patch > > > > > release 1.0.1. > > > > > > > > > > -Marco > > > > > > > > > > Am 24.01.2018 7:20 nachm. schrieb "Zha, Sheng" < > zhasheng@amazon.com > > >: > > > > > > > > > > > There=E2=80=99s an experimental API for text data indexing and = embedding > in > > > > > > mx.contrib.text. > > > > > > > > > > > > - Sent by my thumb > > > > > > > > > > > > > On Jan 24, 2018, at 7:08 PM, Chris Olivier < > > cjolivier01@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > the profiling PR contains a small breaking change, but i don= =E2=80=99t > > > think > > > > > it=E2=80=99s > > > > > > > going into 1.0.1 > > > > > > > > > > > > > >> On Wed, Jan 24, 2018 at 6:48 PM Haibin Lin < > > > > haibin.lin.aws@gmail.com> > > > > > > wrote: > > > > > > >> > > > > > > >> Hi everyone, > > > > > > >> > > > > > > >> Since the plan was to cut a branch from the master branch, the > > > code > > > > > will > > > > > > >> include changes other than the bug fix PRs noted in the > release > > > > note. > > > > > Is > > > > > > >> anyone aware of any API changes in the current MXNet master > > > branch? > > > > In > > > > > > >> particular, are there backward incompatible ones? > > > > > > >> > > > > > > >> Best, > > > > > > >> Haibin > > > > > > >> > > > > > > >> On Tue, Jan 23, 2018 at 11:25 AM, Haibin Lin < > > > > > haibin.lin.aws@gmail.com> > > > > > > >> wrote: > > > > > > >> > > > > > > >>> Hi Sheng, > > > > > > >>> > > > > > > >>> 1. I've been following the discussion on the branching & > > > versioning > > > > > > >>> thread. Features like MKLDNN integration should not go to > patch > > > > > release > > > > > > >>> 1.0.1, and it's risky to merge large PRs right before the > > > release. > > > > > I've > > > > > > >>> removed the MKLDNN section from the release note. > > > > > https://cwiki.apache > > > > > > . > > > > > > >>> org/confluence/display/MXNET/Apache+MXNet+%28incubating%29+ > > > > > > >>> 1.0.1+Release+Notes > > > > > > >>> > > > > > > >>> 2. I agree that we should aim for better test coverage & > stable > > > CI, > > > > > and > > > > > > >>> get those disabled/flaky tests fixed eventually. Fixing these > > > > > requires > > > > > > >>> efforts from the community and I strongly encourage > > contributors > > > to > > > > > > help. > > > > > > >>> Removing the corresponding feature from the release doesn't > > sound > > > > > > >> practical > > > > > > >>> since users might be already using some of those. I suggest > > that > > > we > > > > > > keep > > > > > > >>> track of these tests on Apache Wiki and make sure they are > > > > addressed > > > > > > for > > > > > > >>> the release after 1.0.1. > > > > > > >>> > > > > > > >>> Hi everyone, > > > > > > >>> > > > > > > >>> In terms of the current status for this release, all critical > > bug > > > > > fixes > > > > > > >>> are merged (to the best of my knowledge) and we have made > good > > > > > progress > > > > > > >>> fixing license issues. As Meghna mentioned, a list of open > > > > questions > > > > > > >>> regarding license is at > > > > > > >> https://cwiki.apache.org/confluence/display/MXNET/ > > > > > > >>> MXNet+Source+Licenses section D - it would be great if we can > > get > > > > > more > > > > > > >>> clarification/help/feedback from Apache mentors. > > > > > > >>> > > > > > > >>> I suggest that we shoot for code freeze for 1.0.1 rc0 this > > > > Wednesday. > > > > > > >> Does > > > > > > >>> anyone have concern or objection on this? > > > > > > >>> > > > > > > >>> Best, > > > > > > >>> Haibin > > > > > > >>> > > > > > > >>> On Tue, Jan 23, 2018 at 7:51 AM, Steffen Rochel < > > > > > > steffenrochel@gmail.com > > > > > > >>> > > > > > > >>> wrote: > > > > > > >>> > > > > > > >>>> Hi Sheng - > > > > > > >>>> 1. branch usage and versioning - lets converge our > discussion > > > and > > > > > > >> document > > > > > > >>>> the agreement on wiki. I started a draft summarizing my > > > > > understanding > > > > > > of > > > > > > >>>> the proposal at > > > > > > >>>> https://cwiki.apache.org/confluence/display/MXNET/Release+ > > > > > > >>>> Versioning+and+Branching. > > > > > > >>>> Lets work together to refine and clarify the draft, so we > have > > > > > clarity > > > > > > >>>> going forward. I'm inviting everyone to contribute to this > > > > > discussion. > > > > > > >>>> As MKLDNN integration is not ready yet and we want to > release > > > all > > > > > the > > > > > > >> good > > > > > > >>>> improvements including updates in tutorials and > documentation > > I > > > > > > suggest > > > > > > >> we > > > > > > >>>> move forward with the release asap. As we don't have major > > > > features > > > > > or > > > > > > >>>> non-compatible API changes (to best of my knowledge) I think > > it > > > is > > > > > > >>>> appropriate to label the release as 1.0.1. > > > > > > >>>> Note: This label indicates a patch release. Patch releases > > > should > > > > be > > > > > > >>>> created from the related release branch. As we didn't plan > for > > > it > > > > > and > > > > > > to > > > > > > >>>> minimize overhead I suggest we make a one time exception t= o > > cut > > > > the > > > > > > >> 1.0.1 > > > > > > >>>> release from master branch and clearly communicate in > release > > > > notes. > > > > > > >> Going > > > > > > >>>> forward we should follow the methodology for versioning an= d > > > > > branching > > > > > > to > > > > > > >>>> whatever we agree on. > > > > > > >>>> 2. Disabled tests: I agree with your concerns that we had to > > > > disable > > > > > > 13 > > > > > > >>>> tests due to non-deterministic behavior (see issues > > > > > > >>>> ). > > I'm > > > > > > calling > > > > > > >> on > > > > > > >>>> all contributors to help to resolve the non-deterministic > > > > behavior, > > > > > so > > > > > > >> we > > > > > > >>>> can improve our test coverage. As we discussed offline, lets > > > tests > > > > > > >>>> manually > > > > > > >>>> short term, document the known issue in the release notes > and > > > > > > prioritize > > > > > > >>>> efforts post 1.0.1 release. > > > > > > >>>> > > > > > > >>>> Regards, > > > > > > >>>> Steffen > > > > > > >>>> > > > > > > >>>>> On Wed, Jan 17, 2018 at 5:05 PM Sheng Zha < > > zhasheng@apache.org > > > > > > > > > > wrote: > > > > > > >>>>> > > > > > > >>>>> Hi Haibin, > > > > > > >>>>> > > > > > > >>>>> Thanks for leading this. I suggest that we hold onto this > > > release > > > > > > >> until > > > > > > >>>> we > > > > > > >>>>> have clarity on the following items. > > > > > > >>>>> > > > > > > >>>>> 1. branch usage and versioning > > > > > > >>>>> Given that we are past 1.0 and we're changing APIs, I'd > like > > to > > > > > > >> suggest > > > > > > >>>>> that we first agree on how > > > > > > >>>>> versioning works in mxnet. If we follow semantic > versioning, > > it > > > > > would > > > > > > >>>>> suggest that features like > > > > > > >>>>> MKL-DNN should go at least into 1.1 (minor version change= ) > > > > instead > > > > > of > > > > > > >>>>> 1.0.1 (patch release). > > > > > > >>>>> Also, assuming that new release will come from a new forked > > > > > branch, I > > > > > > >>>>> suggest that we clarify on how to > > > > > > >>>>> name the branches too. > > > > > > >>>>> You can find relevant thread at > > > > > > >>>>> https://lists.apache.org/thread.html/ > c52f8353f63c1e63b2646ec > > > > > > >>>> 3b08d9a8180a1381787d777b41b8ac69f@%3Cdev.mxnet.apache.org > %3E > > > > > > >>>>> > > > > > > >>>>> 2. disabled tests > > > > > > >>>>> For the purpose of stabilizing test automation system, many > > > tests > > > > > > were > > > > > > >>>>> disabled. In order to avoid > > > > > > >>>>> releasing untested features, we should mitigate the > situation > > > of > > > > > > >> having > > > > > > >>>>> disabled tests. > > > > > > >>>>> That means we can fix the tests before the release, or > remove > > > the > > > > > > >>>>> corresponding feature from release > > > > > > >>>>> (might be hard to do, e.g. for optimizer). Otherwise, we > must > > > > > > >>>> collectively > > > > > > >>>>> decide that a feature is > > > > > > >>>>> OK to release without tests. > > > > > > >>>>> The thread on this topic can be found at > > > > > > >>>>> https://lists.apache.org/thread.html/ > addab1937bfcf09b5dfa15c > > > > > > >>>> 1149ddcebd084f1c4bf4e10a73770fb35@%3Cdev.mxnet.apache.org > %3E > > > > > > >>>>> > > > > > > >>>>> We can proceed on the release with more confidence once w= e > > have > > > > > > >> clarity. > > > > > > >>>>> > > > > > > >>>>> Best regards, > > > > > > >>>>> -sz > > > > > > >>>>> > > > > > > >>>>>> On 2018-01-10 15:33, Haibin Lin > > > > > wrote: > > > > > > >>>>>> I am starting the process to prepare for MXNET 1.0.1 > > release. > > > I > > > > > have > > > > > > >>>>>> drafted release notes > > > > > > >>>>>> (* > > > > > > >>>>> https://cwiki.apache.org/confluence/display/MXNET/Apache+ > > > > > > >>>> MXNet+%28incubating%29+1.0.1+Release+Notes > > > > > > >>>>>> < > > > > > > >>>>> https://cwiki.apache.org/confluence/display/MXNET/Apache+ > > > > > > >>>> MXNet+%28incubating%29+1.0.1+Release+Notes > > > > > > >>>>>> *) > > > > > > >>>>>> to cover the tasks under this release. > > > > > > >>>>>> > > > > > > >>>>>> A release candidate will be cut on Monday 22nd Jan, 2018 > and > > > > > voting > > > > > > >>>> will > > > > > > >>>>>> commence from then till Thursday 25th Jan, 2018. If you > have > > > any > > > > > > >>>>> additional > > > > > > >>>>>> features in progress and would like to include it in thi= s > > > > release, > > > > > > >>>> please > > > > > > >>>>>> assure they have been merged by Thursday 18th Jan, 2018 > with > > > > > comment > > > > > > >>>> so I > > > > > > >>>>>> may update the release notes. > > > > > > >>>>>> > > > > > > >>>>>> Feel free to add any other comments/suggestions. > > > > > > >>>>>> > > > > > > >>>>>> Thanks, > > > > > > >>>>>> Haibin > > > > > > >>>>>> > > > > > > >>>>> > > > > > > >>>> > > > > > > >>> > > > > > > >>> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > --94eb2c1ab8ca2b825105639aeaad--