mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tao Lv <ta...@apache.org>
Subject Re: [Discussion] MXNet 1.5.1 release
Date Thu, 29 Aug 2019 15:12:31 GMT
@Aaron,
Thank you for looking into these two issues. I have removed the #15609 from
the scope of 1.5.1. Please let me know if you have any update about #15608.

@Lai,
I'm fine with the decision. License issue about MKL-DNN, cub and pybind is
moved to next release.

@Sam,
I also removed the sidebar issue [3] from the scope of 1.5.1. Besides, I
notice one of your cherry picks is stopped by the CI. Please take a look at
it. Thanks.

*Nice progress since the last update:*
1. Per the discussion, we decided to remove #15609, the license issue about
MKL-DNN, cub and pybind, and the sidebar issue [3] from the scope of 1.5.1
patch release;
2. 3 fixes [4] [5] [6] were merged into the v1.5.x branch.

*Opens (suggested owners are highlighted):*
1. @Aaron is working on #15608 to see if we can have it in v1.5.x;
2. Two cherry pick PRs [7] [8] cannot pass the CI. I have pinged the
authors to take a look at the CI failures.
3. @Kellen proposed 5 fixes [9] for TensorRT but till now only 3 are picked
to v1.5.x. Please help to confirm if the other 2 are still needed.
4. Sorry that I missed the proposal for fixing the nightly build [10] in
previous update. @Lai, can you help to confirm if it's still valid?
5. @Lin please help to make a conclusion for the GPU OOM issue caused by
topk regression [11]. If it cannot be addressed on v1.5.x branch, I will
remove it from the scope of this release and mark it as a known issue in
the release note.

Please find the details in
https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Plan+and+Status
.

Thanks,
-tao

[1] https://github.com/apache/incubator-mxnet/pull/15609
[2] https://github.com/apache/incubator-mxnet/pull/15608
[3] https://github.com/apache/incubator-mxnet/issues/15200
[4] https://github.com/apache/incubator-mxnet/pull/16029
[5] https://github.com/apache/incubator-mxnet/pull/16026
[6] https://github.com/apache/incubator-mxnet/pull/16028
[7] https://github.com/apache/incubator-mxnet/pull/15803
[8] https://github.com/apache/incubator-mxnet/pull/16027
[9]
https://github.com/apache/incubator-mxnet/issues/15613#issuecomment-520688668
[10]
https://github.com/apache/incubator-mxnet/issues/15613#issuecomment-516937546
[11] https://github.com/apache/incubator-mxnet/issues/15703



On Thu, Aug 29, 2019 at 1:06 AM Skalicky, Sam <sskalic@amazon.com.invalid>
wrote:

> Hi Tao,
>
> I just talked with Aaron, lets leave the sidebar issue for later.
>
> I created PRs in the v1.5.x branch to cherry pick the fixes into the 1.5.1
> release:
> https://github.com/apache/incubator-mxnet/pull/16027
> https://github.com/apache/incubator-mxnet/pull/16028
>
> Thanks for your work on this release!
> Sam
>
> On Aug 28, 2019, at 9:35 AM, Lai Wei <royweilai@gmail.com<mailto:
> royweilai@gmail.com>> wrote:
>
> Hi,
>
> Regrading the license issue[1],  we still have item 3, 4, 5 left.
> I think it's better to remove them from 1.5.1 release scope and target for
> 1.6.0 as it need more time and requires changes that should not go into
> patch release.
>
>
> [1] https://github.com/apache/incubator-mxnet/issues/15542
>
> Best Regards
>
> Lai
>
>
> On Wed, Aug 28, 2019 at 9:20 AM Aaron Markham <aaron.s.markham@gmail.com
> <mailto:aaron.s.markham@gmail.com>>
> wrote:
>
> 5 no. Install page defaults to master so you don't need to pick it.
> 6 probably, but there might be other PRs needed. I'd check out the branch
> and attempt the install across platforms to be sure.
>
> On Wed, Aug 28, 2019, 08:55 Tao Lv <taolv@apache.org<mailto:
> taolv@apache.org>> wrote:
>
> Hi Aaron,
>
> They were proposed to be ported to v1.5.x at the beginning of the
> discussion but I didn't see any action for that. So I'm wondering if
> they're still needed. I asked for that in the last update on 8/20 but
> didn't get a response.
>
> If they're still needed, I hope someone who is more familiar with Julia
> frontend can help to cherry pick the commits to the v1.5.x branch.
>
> thanks,
> -tao
>
> On Wed, Aug 28, 2019 at 11:43 PM Aaron Markham <
> aaron.s.markham@gmail.com<mailto:aaron.s.markham@gmail.com>>
> wrote:
>
> I don't see any request for action on the Julia PRs: 5 or 6.
> We didn't put the change in right away because we wanted it to not
> break
> anything. But the changes are needed to make Julia setup more seamless.
>
> What "update" is needed?
>
>
> On Wed, Aug 28, 2019, 08:36 Tao Lv <taolv@apache.org<mailto:
> taolv@apache.org>> wrote:
>
> @Pedro, seems the issue is still open on the master branch. Do you
> still
> think we can have your fix on the 1.5.x branch?
>
> Progress since last update:
> 1. We received several more proposals in the github thread [1]. I
> humbly
> ask the reporters to pick the fixes to the v1.5.x. I will keep
> tracking
> the
> progress and the healthy status of the release branch.
> 2. Thanks to @Lai, the licence issue of julia cat image was fixed on
> the
> master branch and I opened a PR to pick it to v1.5.x [2].
> 3. The GPU OOM issue was fixed on the master branch by @Lin [3] . But
> there
> is a problem with porting the fix to v1.5.x branch [4].
>
> Opens:
> 1. https://github.com/apache/incubator-mxnet/pull/15803 still can
> not
> pass
> the CI;
> 2. Call for a update from julia folks about the back porting for [5]
> and
> [6]
> 3. License issue of cub and pybind is still open. @Lai opened a PR
> [7]
> to
> update cub submodule but seems it need more effort than just commit
> id
> update. I suspect that we cannot finish this work in 1.5.1 patch
> release.
> 4. Still no progress for the sidebar issue on web page [8].
> 5. Call for a conclusion about fixing the GPU OOM issue in 1.5.1
>
> Besides, I would like to ask if there is any preference for the
> release
> timeline of 1.5.1 patch release? Please share so I can propose the
> time
> for
> code freeze.
>
> Thanks,
> -tao
>
> [1]  https://github.com/apache/incubator-mxnet/issues/15613.
> [2] https://github.com/apache/incubator-mxnet/pull/16026
> [3] https://github.com/apache/incubator-mxnet/pull/15948
> [4] https://github.com/apache/incubator-mxnet/pull/15999
> [5] https://github.com/apache/incubator-mxnet/pull/15609
> [6]  https://github.com/apache/incubator-mxnet/pull/15608
> [7] https://github.com/apache/incubator-mxnet/pull/15963
> [8] https://github.com/apache/incubator-mxnet/issues/15200
>
> On Wed, Aug 28, 2019 at 5:50 AM Pedro Larroy <
> pedro.larroy.lists@gmail.com<mailto:pedro.larroy.lists@gmail.com>
>
> wrote:
>
> Ok. I was just asking if we want this fix in 1.5.1 since it
> addresses
> crashes using multiprocessing. The problem with cherry picking is
> that
> the
> patch contains the dynamic load change which shouldn't impact
> anything
> else
> but is not supposed to go in a release branch.
>
> On Tue, Aug 27, 2019 at 1:19 PM Lin Yuan <apeforest@gmail.com<mailto:
> apeforest@gmail.com>>
> wrote:
>
> https://github.com/apache/incubator-mxnet/pull/15762  contains
> some
> unrelated changes which is being reverted. Please do not cherry
> pick
> it
> yet.
>
> On Mon, Aug 26, 2019 at 4:25 PM Pedro Larroy <
> pedro.larroy.lists@gmail.com<mailto:pedro.larroy.lists@gmail.com>
>
> wrote:
>
> There's a fix that I did which seems to still produce crashes
> in
> 1.5
> for
> some users, which I got notice today and is fixed in master.
>
> Might be useful to put in 1.5.1:
> https://github.com/apache/incubator-mxnet/pull/15762   ?
>
> Pedro.
>
> On Tue, Aug 20, 2019 at 7:49 AM Tao Lv <taolv@apache.org<mailto:
> taolv@apache.org>>
> wrote:
>
> Hi dev,
>
> Here is an update for the 1.5.1 patch release.
>
> 1. Thanks for the effort from whole community, we have cherry
> picked
> a
> bunch of fixes to v1.5.x branch. So far, the branch looks
> healthy:
>
>
>
>
>
>
>
>
>
> http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/activity/
> 2. https://github.com/apache/incubator-mxnet/pull/15803
> cannot
> pass
> the
> CI;
> 3. I hope julia folks can take a look at the back porting for
> https://github.com/apache/incubator-mxnet/pull/15609 and
> https://github.com/apache/incubator-mxnet/pull/15608 - do we
> still
> need
> them?
> 4. License issue of cub and pybind is still not fixed. We
> also
> has
> a
> license issue of a cat image in julia examples.
> https://github.com/apache/incubator-mxnet/issues/15542
> 5. Still no progress for the sidebar issue:
> https://github.com/apache/incubator-mxnet/issues/15200
> 6. There is a GPU OOM issue in 1.5.0 release and already root
> caused
> by
> Lin:
>
>
>
>
>
>
>
>
>
> https://github.com/apache/incubator-mxnet/issues/15703#issuecomment-522780492
> .
> We need decide whether we want to get it fixed in the 1.5.1
> patch
> release.
>
> Please find details in
>
>
>
>
>
>
>
>
>
> https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Plan+and+Status
> .
>
> Thanks,
> -tao
>
> On Mon, Aug 12, 2019 at 9:57 PM Zhao, Patric <
> patric.zhao@intel.com>
> wrote:
>
> Thanks for the explanation, Marco & Tao. Sounds great!
>
> -----Original Message-----
> From: Tao Lv <taolv@apache.org>
> Sent: Monday, August 12, 2019 9:54 PM
> To: dev@mxnet.incubator.apache.org
> Subject: Re: [Discussion] MXNet 1.5.1 release
>
> Regarding the open issue, is there default code
> owner/maintainer?
> If
> so, he/she will be the right people to look into the
> issue.
>
> https://github.com/apache/incubator-mxnet/blob/master/CODEOWNERS
>
>
> I have no idea. But the CODEOWNERS is used to receive
> change
> notificaitons,
> not actually indicates the maintainer of a piece of code.
>
> Do we have regularly build, run, functionality and
> performance
> testing
> for
> this release?
>
>
> As Marco mentioned, build, run and functionality of
> v1.5.x
> branch
> are
> tracked
> automatically by the CI for each cherry pick pull request
> and
> the
> nightly tests
> here:
> http://jenkins.mxnet-ci.amazon-
>
> ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/activity
> .
> I see it's healthy so far.
>
> For performance, Shufan will track CPU performance with
> his
> test
> suite
> and
> send out the report once the branch is frozen. I'm not
> sure
> if
> there
> are
> any
> other performance tests.
>
> On Mon, Aug 12, 2019 at 9:36 PM Marco de Abreu
> <marco.g.abreu@gmail.com>
> wrote:
>
> Hi Patric,
>
> CI should automatically pick up the branch and validate
> it
> as
> usual.
>
> Best regards,
> Marco
>
> Zhao, Patric <patric.zhao@intel.com> schrieb am Mo.,
> 12.
> Aug.
> 2019,
> 15:22:
>
> It's great works, Tao 😊
>
> Regarding the open issue, is there default code
> owner/maintainer?
> If
> so, he/she will be the right people to look into the
> issue.
> https://github.com/apache/incubator-
> mxnet/blob/master/CODEOWNERS
>
> Do we have regularly build, run, functionality and
> performance
> testing
> for
> this release?
>
> Thanks,
>
> --Patric
>
> -----Original Message-----
> From: Tao Lv <taolv@apache.org>
> Sent: Monday, August 12, 2019 8:59 PM
> To: dev@mxnet.incubator.apache.org
> Subject: Re: [Discussion] MXNet 1.5.1 release
>
> Update:
>
> We're cherry picking fixes from the master to the
> v1.5.x
> branch.
> Some
> of
> them are already merged. Please find details on the
> cwiki
> page:
>
>
> https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl
> an+a
> nd+Status
>
>
> There are still 3 opens:
> 1. Nightly test failure on CI (
>
> https://github.com/apache/incubator-mxnet/issues/15374
> ):
> The
> issue
> is
> still
> open. I'm wondering if it has been fixed or not. If
> not,
> is
> there
> anyone
> working on it?
> 2. Broken Sidebar on website API for master and
> 1.5.0 (
>
> https://github.com/apache/incubator-mxnet/issues/15200
> ):
> I
> don't
> see
> any
> progress on this issue? Do we still want to include
> it
> into
> 1.5.1
> patch
> release?
> 3. License issues need to be fixed before 1.6
> release (
>
> https://github.com/apache/incubator-mxnet/issues/15542
> ):
> Currently
> the license issue for code and images is partially
> fixed
> on
> the
> master
> branch and
> will be picked to v1.5.x soon. MKLML license issue
> is
> pushed
> out
> to 1.6 release. But license issue for cub and
> pybind
> is
> still
> open.
>
> Let me know if you any suggestion. Thanks for your
> support!
>
> -tao
>
>
> On Wed, Aug 7, 2019 at 11:03 PM Tao Lv <
> taolv@apache.org
>
> wrote:
>
>
> Update:
>
> Thanks to wkcn's report, Issue #15774 [1] and the
> fix
> #15751
> [2]
> are added to the scope of 1.5.1 patch release.
> For issue #15703 [3], I'm still waiting from the
> response
> from
> the reporter.
> Issue #15431 [4] was closed as false positive
> report.
> I also included several MKL-DNN backend issues
> reported
> by
> mxnet
> users
> and downstream projects. They are already fixed
> on
> the
> master
> branch.
>
> Please kindly check the full list of issues need
> be
> included
> in
> the
> 1.5.1 patch release:
>
>
>
> https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl
> an+a
> nd+Status
>
> For issues which are already fixed on the master
> branch,
> we
> will
> start
> to cherry pick the fix commit to the v1.5.x
> branch.
> For
> issues
> which are still open, we will start to track the
> fix
> process.
>
> Thanks for your great support. Let me know if you
> have
> any
> questions or concerns.
>
> -tao
>
> [1]
> https://github.com/apache/incubator-mxnet/issues/15774
> [2]
> https://github.com/apache/incubator-mxnet/pull/15751
> [3]
> https://github.com/apache/incubator-mxnet/issues/15703
> [4]
> https://github.com/apache/incubator-mxnet/issues/15431
>
>
> On Tue, Aug 6, 2019 at 2:04 PM Tao Lv <
> taolv@apache.org>
> wrote:
>
>
> Per Sam's proposal [1], Issue #15737 [2] and the
> fix
> [3]
> are
> added
> to
> the scope of 1.5.1 patch release.
>
> A friendly reminder: the issue proposing will be
> closed
> before
> 11pm
> 8/7 CST (8am 8/7 PST). After that, we will start
> to
> cherry
> pick
> fixes
> to the v1.5.x branch.
>
>
> [1]
> https://github.com/apache/incubator-
> mxnet/issues/15613#issuecomment-5
> 18430120 [2]
>
> https://github.com/apache/incubator-mxnet/issues/15737
> [3]
> https://github.com/apache/incubator-mxnet/pull/15692
>
> On Thu, Aug 1, 2019 at 4:24 PM Tao Lv <
> taolv@apache.org
>
> wrote:
>
> Hi Sandeep/Lai,
>
> Thank you for the prompt response!
>
>
> https://github.com/apache/incubator-mxnet/issues/15200
> is
> added
> to
> the list to track the sidebar issue.
>
> On Thu, Aug 1, 2019 at 7:54 AM sandeep
> krishnamurthy
> <
> sandeep.krishna98@gmail.com> wrote:
>
> Thank you Tao and Shufan.
> Sidebar missing bug in API documentation is
> inconvenience
> for
> the
> user.
> It
> would great if we can fix it with 1.5.1
>
> On Wed, Jul 31, 2019, 10:14 AM Lai Wei <
> royweilai@gmail.com
>
> wrote:
>
> Hi Tao,
>
> Thank you so much for driving it.  Currently
> nightly
> test
> on
> tutorials are
> failing and it need to be fixed. [3] I have
> updated
> the
> issue[1] and cwiki.[2]
>
> [1]
> https://github.com/apache/incubator-mxnet/issues/15613
> [2]
>
>
>
>
>
> https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl
> a
> n+and+Status
> [3]
> https://github.com/apache/incubator-mxnet/issues/15374
>
> Best Regards
>
> Lai
>
>
> On Wed, Jul 31, 2019 at 8:04 AM Tao Lv <
> taolv@apache.org>
> wrote:
>
> Hi community,
>
>
>
> Thanks for the initiative from Sam
> (samskalicky@github
> ),
> we already
> have a
> discussion thread [1] on github about the
> defects
> and
> bugs exposed
> in the
> 1.5.0 release.
>
> Shufan (juliusshufan@github) and I
> (TaoLv@github)
> would
> like
> to
> manage
> the
> release of 1.5.1. This will be our first
> debut
> on
> the
> release
> process,
> your
> comments are always valuable.
>
>
>
> Per the SemVer 2.0 [2], MXNet 1.5.1 will
> be
> a
> patch
> release which
> contains
> backwards-compatible fixes only.
>
> I have created a page on cwiki [3] to
> track
> the
> release
> process
> and
> moved
> the issues and PRs mentioned in the github
> discussion
> thread
> to
> the
> page.
>
>
>
> Here I would like to ask the community to:
>
> (1) Raise any other defect or regression
> you
> identified
> in the
> 1.5.0 release. Please file a github issue
> for
> it
> and
> note
> the issue
> number in
> this thread;
>
> (2) Please comment with one sentence for
> why
> you
> think
> the issue is critical and must have in the
> 1.5.1
> release;
>
> (3) If the issue is already fixed on
> master
> branch
> or
> already have
> a PR
> WIP, please also note the fix commit id or
> PR
> number;
>
> (4) If the issue is still open and there
> is
> no
> PR
> WIP,
> please
> indicate
> whether you'd be willing to help it out;
>
> (5) Feel free to comment if any other
> suggestion
> for
> the
> release.
>
>
>
> I suggest to keep this thread open for one
> week
> to
> collect enough information and proposals
> before
> we
> decide
> the timeline for the
> release.
> So
> your timely response will be highly
> appreciated!
>
>
>
> PS: Sorry to say that even as a committer,
> this
> is
> the
> first time
> for me
> to
> manage a release. So it would be great if
> an
> experienced
> committer
> can
> help
> to guide the process.
>
>
>
> -tao
>
>
>
> [1]
>
> https://github.com/apache/incubator-mxnet/issues/15613
>
> [2] https://semver.org/
>
> [3]
>
>
>
>
>
>
> https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl
> a
> n+and+Status
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message