mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lai Wei <roywei...@gmail.com>
Subject Re: [Discussion] MXNet 1.5.1 release
Date Thu, 29 Aug 2019 23:15:58 GMT
Hi Tao,

Just checked 1.5.x nightly build is passing, so 10 is not needed. I moved
it so 1.6.0 scope.

Thanks


Best Regards

Lai


On Thu, Aug 29, 2019 at 8:12 AM Tao Lv <taolv@apache.org> wrote:

> @Aaron,
> Thank you for looking into these two issues. I have removed the #15609 from
> the scope of 1.5.1. Please let me know if you have any update about #15608.
>
> @Lai,
> I'm fine with the decision. License issue about MKL-DNN, cub and pybind is
> moved to next release.
>
> @Sam,
> I also removed the sidebar issue [3] from the scope of 1.5.1. Besides, I
> notice one of your cherry picks is stopped by the CI. Please take a look at
> it. Thanks.
>
> *Nice progress since the last update:*
> 1. Per the discussion, we decided to remove #15609, the license issue about
> MKL-DNN, cub and pybind, and the sidebar issue [3] from the scope of 1.5.1
> patch release;
> 2. 3 fixes [4] [5] [6] were merged into the v1.5.x branch.
>
> *Opens (suggested owners are highlighted):*
> 1. @Aaron is working on #15608 to see if we can have it in v1.5.x;
> 2. Two cherry pick PRs [7] [8] cannot pass the CI. I have pinged the
> authors to take a look at the CI failures.
> 3. @Kellen proposed 5 fixes [9] for TensorRT but till now only 3 are picked
> to v1.5.x. Please help to confirm if the other 2 are still needed.
> 4. Sorry that I missed the proposal for fixing the nightly build [10] in
> previous update. @Lai, can you help to confirm if it's still valid?
> 5. @Lin please help to make a conclusion for the GPU OOM issue caused by
> topk regression [11]. If it cannot be addressed on v1.5.x branch, I will
> remove it from the scope of this release and mark it as a known issue in
> the release note.
>
> Please find the details in
>
> https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Plan+and+Status
> .
>
> Thanks,
> -tao
>
> [1] https://github.com/apache/incubator-mxnet/pull/15609
> [2] https://github.com/apache/incubator-mxnet/pull/15608
> [3] https://github.com/apache/incubator-mxnet/issues/15200
> [4] https://github.com/apache/incubator-mxnet/pull/16029
> [5] https://github.com/apache/incubator-mxnet/pull/16026
> [6] https://github.com/apache/incubator-mxnet/pull/16028
> [7] https://github.com/apache/incubator-mxnet/pull/15803
> [8] https://github.com/apache/incubator-mxnet/pull/16027
> [9]
>
> https://github.com/apache/incubator-mxnet/issues/15613#issuecomment-520688668
> [10]
>
> https://github.com/apache/incubator-mxnet/issues/15613#issuecomment-516937546
> [11] https://github.com/apache/incubator-mxnet/issues/15703
>
>
>
> On Thu, Aug 29, 2019 at 1:06 AM Skalicky, Sam <sskalic@amazon.com.invalid>
> wrote:
>
> > Hi Tao,
> >
> > I just talked with Aaron, lets leave the sidebar issue for later.
> >
> > I created PRs in the v1.5.x branch to cherry pick the fixes into the
> 1.5.1
> > release:
> > https://github.com/apache/incubator-mxnet/pull/16027
> > https://github.com/apache/incubator-mxnet/pull/16028
> >
> > Thanks for your work on this release!
> > Sam
> >
> > On Aug 28, 2019, at 9:35 AM, Lai Wei <royweilai@gmail.com<mailto:
> > royweilai@gmail.com>> wrote:
> >
> > Hi,
> >
> > Regrading the license issue[1],  we still have item 3, 4, 5 left.
> > I think it's better to remove them from 1.5.1 release scope and target
> for
> > 1.6.0 as it need more time and requires changes that should not go into
> > patch release.
> >
> >
> > [1] https://github.com/apache/incubator-mxnet/issues/15542
> >
> > Best Regards
> >
> > Lai
> >
> >
> > On Wed, Aug 28, 2019 at 9:20 AM Aaron Markham <aaron.s.markham@gmail.com
> > <mailto:aaron.s.markham@gmail.com>>
> > wrote:
> >
> > 5 no. Install page defaults to master so you don't need to pick it.
> > 6 probably, but there might be other PRs needed. I'd check out the branch
> > and attempt the install across platforms to be sure.
> >
> > On Wed, Aug 28, 2019, 08:55 Tao Lv <taolv@apache.org<mailto:
> > taolv@apache.org>> wrote:
> >
> > Hi Aaron,
> >
> > They were proposed to be ported to v1.5.x at the beginning of the
> > discussion but I didn't see any action for that. So I'm wondering if
> > they're still needed. I asked for that in the last update on 8/20 but
> > didn't get a response.
> >
> > If they're still needed, I hope someone who is more familiar with Julia
> > frontend can help to cherry pick the commits to the v1.5.x branch.
> >
> > thanks,
> > -tao
> >
> > On Wed, Aug 28, 2019 at 11:43 PM Aaron Markham <
> > aaron.s.markham@gmail.com<mailto:aaron.s.markham@gmail.com>>
> > wrote:
> >
> > I don't see any request for action on the Julia PRs: 5 or 6.
> > We didn't put the change in right away because we wanted it to not
> > break
> > anything. But the changes are needed to make Julia setup more seamless.
> >
> > What "update" is needed?
> >
> >
> > On Wed, Aug 28, 2019, 08:36 Tao Lv <taolv@apache.org<mailto:
> > taolv@apache.org>> wrote:
> >
> > @Pedro, seems the issue is still open on the master branch. Do you
> > still
> > think we can have your fix on the 1.5.x branch?
> >
> > Progress since last update:
> > 1. We received several more proposals in the github thread [1]. I
> > humbly
> > ask the reporters to pick the fixes to the v1.5.x. I will keep
> > tracking
> > the
> > progress and the healthy status of the release branch.
> > 2. Thanks to @Lai, the licence issue of julia cat image was fixed on
> > the
> > master branch and I opened a PR to pick it to v1.5.x [2].
> > 3. The GPU OOM issue was fixed on the master branch by @Lin [3] . But
> > there
> > is a problem with porting the fix to v1.5.x branch [4].
> >
> > Opens:
> > 1. https://github.com/apache/incubator-mxnet/pull/15803 still can
> > not
> > pass
> > the CI;
> > 2. Call for a update from julia folks about the back porting for [5]
> > and
> > [6]
> > 3. License issue of cub and pybind is still open. @Lai opened a PR
> > [7]
> > to
> > update cub submodule but seems it need more effort than just commit
> > id
> > update. I suspect that we cannot finish this work in 1.5.1 patch
> > release.
> > 4. Still no progress for the sidebar issue on web page [8].
> > 5. Call for a conclusion about fixing the GPU OOM issue in 1.5.1
> >
> > Besides, I would like to ask if there is any preference for the
> > release
> > timeline of 1.5.1 patch release? Please share so I can propose the
> > time
> > for
> > code freeze.
> >
> > Thanks,
> > -tao
> >
> > [1]  https://github.com/apache/incubator-mxnet/issues/15613.
> > [2] https://github.com/apache/incubator-mxnet/pull/16026
> > [3] https://github.com/apache/incubator-mxnet/pull/15948
> > [4] https://github.com/apache/incubator-mxnet/pull/15999
> > [5] https://github.com/apache/incubator-mxnet/pull/15609
> > [6]  https://github.com/apache/incubator-mxnet/pull/15608
> > [7] https://github.com/apache/incubator-mxnet/pull/15963
> > [8] https://github.com/apache/incubator-mxnet/issues/15200
> >
> > On Wed, Aug 28, 2019 at 5:50 AM Pedro Larroy <
> > pedro.larroy.lists@gmail.com<mailto:pedro.larroy.lists@gmail.com>
> >
> > wrote:
> >
> > Ok. I was just asking if we want this fix in 1.5.1 since it
> > addresses
> > crashes using multiprocessing. The problem with cherry picking is
> > that
> > the
> > patch contains the dynamic load change which shouldn't impact
> > anything
> > else
> > but is not supposed to go in a release branch.
> >
> > On Tue, Aug 27, 2019 at 1:19 PM Lin Yuan <apeforest@gmail.com<mailto:
> > apeforest@gmail.com>>
> > wrote:
> >
> > https://github.com/apache/incubator-mxnet/pull/15762  contains
> > some
> > unrelated changes which is being reverted. Please do not cherry
> > pick
> > it
> > yet.
> >
> > On Mon, Aug 26, 2019 at 4:25 PM Pedro Larroy <
> > pedro.larroy.lists@gmail.com<mailto:pedro.larroy.lists@gmail.com>
> >
> > wrote:
> >
> > There's a fix that I did which seems to still produce crashes
> > in
> > 1.5
> > for
> > some users, which I got notice today and is fixed in master.
> >
> > Might be useful to put in 1.5.1:
> > https://github.com/apache/incubator-mxnet/pull/15762   ?
> >
> > Pedro.
> >
> > On Tue, Aug 20, 2019 at 7:49 AM Tao Lv <taolv@apache.org<mailto:
> > taolv@apache.org>>
> > wrote:
> >
> > Hi dev,
> >
> > Here is an update for the 1.5.1 patch release.
> >
> > 1. Thanks for the effort from whole community, we have cherry
> > picked
> > a
> > bunch of fixes to v1.5.x branch. So far, the branch looks
> > healthy:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/activity/
> > 2. https://github.com/apache/incubator-mxnet/pull/15803
> > cannot
> > pass
> > the
> > CI;
> > 3. I hope julia folks can take a look at the back porting for
> > https://github.com/apache/incubator-mxnet/pull/15609 and
> > https://github.com/apache/incubator-mxnet/pull/15608 - do we
> > still
> > need
> > them?
> > 4. License issue of cub and pybind is still not fixed. We
> > also
> > has
> > a
> > license issue of a cat image in julia examples.
> > https://github.com/apache/incubator-mxnet/issues/15542
> > 5. Still no progress for the sidebar issue:
> > https://github.com/apache/incubator-mxnet/issues/15200
> > 6. There is a GPU OOM issue in 1.5.0 release and already root
> > caused
> > by
> > Lin:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> https://github.com/apache/incubator-mxnet/issues/15703#issuecomment-522780492
> > .
> > We need decide whether we want to get it fixed in the 1.5.1
> > patch
> > release.
> >
> > Please find details in
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Plan+and+Status
> > .
> >
> > Thanks,
> > -tao
> >
> > On Mon, Aug 12, 2019 at 9:57 PM Zhao, Patric <
> > patric.zhao@intel.com>
> > wrote:
> >
> > Thanks for the explanation, Marco & Tao. Sounds great!
> >
> > -----Original Message-----
> > From: Tao Lv <taolv@apache.org>
> > Sent: Monday, August 12, 2019 9:54 PM
> > To: dev@mxnet.incubator.apache.org
> > Subject: Re: [Discussion] MXNet 1.5.1 release
> >
> > Regarding the open issue, is there default code
> > owner/maintainer?
> > If
> > so, he/she will be the right people to look into the
> > issue.
> >
> > https://github.com/apache/incubator-mxnet/blob/master/CODEOWNERS
> >
> >
> > I have no idea. But the CODEOWNERS is used to receive
> > change
> > notificaitons,
> > not actually indicates the maintainer of a piece of code.
> >
> > Do we have regularly build, run, functionality and
> > performance
> > testing
> > for
> > this release?
> >
> >
> > As Marco mentioned, build, run and functionality of
> > v1.5.x
> > branch
> > are
> > tracked
> > automatically by the CI for each cherry pick pull request
> > and
> > the
> > nightly tests
> > here:
> > http://jenkins.mxnet-ci.amazon-
> >
> > ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/activity
> > .
> > I see it's healthy so far.
> >
> > For performance, Shufan will track CPU performance with
> > his
> > test
> > suite
> > and
> > send out the report once the branch is frozen. I'm not
> > sure
> > if
> > there
> > are
> > any
> > other performance tests.
> >
> > On Mon, Aug 12, 2019 at 9:36 PM Marco de Abreu
> > <marco.g.abreu@gmail.com>
> > wrote:
> >
> > Hi Patric,
> >
> > CI should automatically pick up the branch and validate
> > it
> > as
> > usual.
> >
> > Best regards,
> > Marco
> >
> > Zhao, Patric <patric.zhao@intel.com> schrieb am Mo.,
> > 12.
> > Aug.
> > 2019,
> > 15:22:
> >
> > It's great works, Tao 😊
> >
> > Regarding the open issue, is there default code
> > owner/maintainer?
> > If
> > so, he/she will be the right people to look into the
> > issue.
> > https://github.com/apache/incubator-
> > mxnet/blob/master/CODEOWNERS
> >
> > Do we have regularly build, run, functionality and
> > performance
> > testing
> > for
> > this release?
> >
> > Thanks,
> >
> > --Patric
> >
> > -----Original Message-----
> > From: Tao Lv <taolv@apache.org>
> > Sent: Monday, August 12, 2019 8:59 PM
> > To: dev@mxnet.incubator.apache.org
> > Subject: Re: [Discussion] MXNet 1.5.1 release
> >
> > Update:
> >
> > We're cherry picking fixes from the master to the
> > v1.5.x
> > branch.
> > Some
> > of
> > them are already merged. Please find details on the
> > cwiki
> > page:
> >
> >
> > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl
> > an+a
> > nd+Status
> >
> >
> > There are still 3 opens:
> > 1. Nightly test failure on CI (
> >
> > https://github.com/apache/incubator-mxnet/issues/15374
> > ):
> > The
> > issue
> > is
> > still
> > open. I'm wondering if it has been fixed or not. If
> > not,
> > is
> > there
> > anyone
> > working on it?
> > 2. Broken Sidebar on website API for master and
> > 1.5.0 (
> >
> > https://github.com/apache/incubator-mxnet/issues/15200
> > ):
> > I
> > don't
> > see
> > any
> > progress on this issue? Do we still want to include
> > it
> > into
> > 1.5.1
> > patch
> > release?
> > 3. License issues need to be fixed before 1.6
> > release (
> >
> > https://github.com/apache/incubator-mxnet/issues/15542
> > ):
> > Currently
> > the license issue for code and images is partially
> > fixed
> > on
> > the
> > master
> > branch and
> > will be picked to v1.5.x soon. MKLML license issue
> > is
> > pushed
> > out
> > to 1.6 release. But license issue for cub and
> > pybind
> > is
> > still
> > open.
> >
> > Let me know if you any suggestion. Thanks for your
> > support!
> >
> > -tao
> >
> >
> > On Wed, Aug 7, 2019 at 11:03 PM Tao Lv <
> > taolv@apache.org
> >
> > wrote:
> >
> >
> > Update:
> >
> > Thanks to wkcn's report, Issue #15774 [1] and the
> > fix
> > #15751
> > [2]
> > are added to the scope of 1.5.1 patch release.
> > For issue #15703 [3], I'm still waiting from the
> > response
> > from
> > the reporter.
> > Issue #15431 [4] was closed as false positive
> > report.
> > I also included several MKL-DNN backend issues
> > reported
> > by
> > mxnet
> > users
> > and downstream projects. They are already fixed
> > on
> > the
> > master
> > branch.
> >
> > Please kindly check the full list of issues need
> > be
> > included
> > in
> > the
> > 1.5.1 patch release:
> >
> >
> >
> > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl
> > an+a
> > nd+Status
> >
> > For issues which are already fixed on the master
> > branch,
> > we
> > will
> > start
> > to cherry pick the fix commit to the v1.5.x
> > branch.
> > For
> > issues
> > which are still open, we will start to track the
> > fix
> > process.
> >
> > Thanks for your great support. Let me know if you
> > have
> > any
> > questions or concerns.
> >
> > -tao
> >
> > [1]
> > https://github.com/apache/incubator-mxnet/issues/15774
> > [2]
> > https://github.com/apache/incubator-mxnet/pull/15751
> > [3]
> > https://github.com/apache/incubator-mxnet/issues/15703
> > [4]
> > https://github.com/apache/incubator-mxnet/issues/15431
> >
> >
> > On Tue, Aug 6, 2019 at 2:04 PM Tao Lv <
> > taolv@apache.org>
> > wrote:
> >
> >
> > Per Sam's proposal [1], Issue #15737 [2] and the
> > fix
> > [3]
> > are
> > added
> > to
> > the scope of 1.5.1 patch release.
> >
> > A friendly reminder: the issue proposing will be
> > closed
> > before
> > 11pm
> > 8/7 CST (8am 8/7 PST). After that, we will start
> > to
> > cherry
> > pick
> > fixes
> > to the v1.5.x branch.
> >
> >
> > [1]
> > https://github.com/apache/incubator-
> > mxnet/issues/15613#issuecomment-5
> > 18430120 [2]
> >
> > https://github.com/apache/incubator-mxnet/issues/15737
> > [3]
> > https://github.com/apache/incubator-mxnet/pull/15692
> >
> > On Thu, Aug 1, 2019 at 4:24 PM Tao Lv <
> > taolv@apache.org
> >
> > wrote:
> >
> > Hi Sandeep/Lai,
> >
> > Thank you for the prompt response!
> >
> >
> > https://github.com/apache/incubator-mxnet/issues/15200
> > is
> > added
> > to
> > the list to track the sidebar issue.
> >
> > On Thu, Aug 1, 2019 at 7:54 AM sandeep
> > krishnamurthy
> > <
> > sandeep.krishna98@gmail.com> wrote:
> >
> > Thank you Tao and Shufan.
> > Sidebar missing bug in API documentation is
> > inconvenience
> > for
> > the
> > user.
> > It
> > would great if we can fix it with 1.5.1
> >
> > On Wed, Jul 31, 2019, 10:14 AM Lai Wei <
> > royweilai@gmail.com
> >
> > wrote:
> >
> > Hi Tao,
> >
> > Thank you so much for driving it.  Currently
> > nightly
> > test
> > on
> > tutorials are
> > failing and it need to be fixed. [3] I have
> > updated
> > the
> > issue[1] and cwiki.[2]
> >
> > [1]
> > https://github.com/apache/incubator-mxnet/issues/15613
> > [2]
> >
> >
> >
> >
> >
> > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl
> > a
> > n+and+Status
> > [3]
> > https://github.com/apache/incubator-mxnet/issues/15374
> >
> > Best Regards
> >
> > Lai
> >
> >
> > On Wed, Jul 31, 2019 at 8:04 AM Tao Lv <
> > taolv@apache.org>
> > wrote:
> >
> > Hi community,
> >
> >
> >
> > Thanks for the initiative from Sam
> > (samskalicky@github
> > ),
> > we already
> > have a
> > discussion thread [1] on github about the
> > defects
> > and
> > bugs exposed
> > in the
> > 1.5.0 release.
> >
> > Shufan (juliusshufan@github) and I
> > (TaoLv@github)
> > would
> > like
> > to
> > manage
> > the
> > release of 1.5.1. This will be our first
> > debut
> > on
> > the
> > release
> > process,
> > your
> > comments are always valuable.
> >
> >
> >
> > Per the SemVer 2.0 [2], MXNet 1.5.1 will
> > be
> > a
> > patch
> > release which
> > contains
> > backwards-compatible fixes only.
> >
> > I have created a page on cwiki [3] to
> > track
> > the
> > release
> > process
> > and
> > moved
> > the issues and PRs mentioned in the github
> > discussion
> > thread
> > to
> > the
> > page.
> >
> >
> >
> > Here I would like to ask the community to:
> >
> > (1) Raise any other defect or regression
> > you
> > identified
> > in the
> > 1.5.0 release. Please file a github issue
> > for
> > it
> > and
> > note
> > the issue
> > number in
> > this thread;
> >
> > (2) Please comment with one sentence for
> > why
> > you
> > think
> > the issue is critical and must have in the
> > 1.5.1
> > release;
> >
> > (3) If the issue is already fixed on
> > master
> > branch
> > or
> > already have
> > a PR
> > WIP, please also note the fix commit id or
> > PR
> > number;
> >
> > (4) If the issue is still open and there
> > is
> > no
> > PR
> > WIP,
> > please
> > indicate
> > whether you'd be willing to help it out;
> >
> > (5) Feel free to comment if any other
> > suggestion
> > for
> > the
> > release.
> >
> >
> >
> > I suggest to keep this thread open for one
> > week
> > to
> > collect enough information and proposals
> > before
> > we
> > decide
> > the timeline for the
> > release.
> > So
> > your timely response will be highly
> > appreciated!
> >
> >
> >
> > PS: Sorry to say that even as a committer,
> > this
> > is
> > the
> > first time
> > for me
> > to
> > manage a release. So it would be great if
> > an
> > experienced
> > committer
> > can
> > help
> > to guide the process.
> >
> >
> >
> > -tao
> >
> >
> >
> > [1]
> >
> > https://github.com/apache/incubator-mxnet/issues/15613
> >
> > [2] https://semver.org/
> >
> > [3]
> >
> >
> >
> >
> >
> >
> > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl
> > a
> > n+and+Status
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message