mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tao Lv <mutou...@gmail.com>
Subject Re: Stopping nightly releases to Pypi
Date Tue, 03 Dec 2019 13:28:10 GMT
As we have many users in China, I'm considering the accessibility of S3.
For pip, we can mirrors.

On Tue, Dec 3, 2019 at 3:24 PM Lausen, Leonard <lausen@amazon.com.invalid>
wrote:

> I would like to remind everyone that lazy consensus is assumed if no
> objections
> are raised before 2019-12-05 at 05:42 UTC. There has been some discussion
> about
> the proposal, but to my understanding no objections were raised.
>
> If the proposal is accepted, MXNet releases would be installed via
>
>    pip install mxnet
>
> And release candidates via
>
>   pip install --pre mxnet
>
> (or with the respective cuda version specifier appended etc.)
>
> To obtain releases built automatically from the master branch, users would
> need
> to specify something like "-f
> http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html" option to pip.
>
> Best regards
> Leonard
>
> On Mon, 2019-12-02 at 05:42 +0000, Lausen, Leonard wrote:
> > Hi MXNet Community,
> >
> > since more than 2 months our binary Python nightly releases published on
> Pypi
> > are broken. The problem is that our binaries exceed Pypi's size limit.
> > Decreasing the binary size by adding compression breaks third-party
> libraries
> > loading libmxnet.so
> https://github.com/apache/incubator-mxnet/issues/16193
> >
> > Sheng requested Pypi to increase their size limit:
> > https://github.com/pypa/pypi-support/issues/50
> >
> > Currently "the biggest cost for PyPI from [the many MXNet binaries with
> > nightly
> > release to Pypi] is the bandwidth consumed when several hundred mirrors
> > attempt
> > to mirror each release immediately after it's published". So Pypi is not
> > inclined to allow us to upload even larger binaries on a nightly
> schedule.
> > Their compromise is to allow it on a weekly cadence.
> >
> > However, I would like the community to revisit the necessity of
> releasing pre-
> > release binaries to Pypi on a nightly (or weekly) cadence. Instead, we
> can
> > release nightly binaries ONLY to a public S3 bucket and instruct users to
> > install from there. On our side, we only need to prepare a html document
> that
> > contains links to all released nightly binaries.
> > Finally users will install the nightly releases via
> >
> >   pip install --pre mxnet-cu101 -f
> http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > nightly.html
> >
> > Instead of
> >
> >   pip install --pre mxnet-cu101
> >
> > Of course proper releases and release candidates should still be made
> > available
> > via Pypi. Thus releases would be installed via
> >
> >   pip install mxnet-cu101
> >
> > And release candidates via
> >
> >   pip install --pre mxnet-cu101
> >
> > This will substantially reduce the costs of the Pypi project and in fact
> > matches
> > the installation experience provided by PyTorch. I don't think the
> benefit of
> > not including "-f http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html
> "
> > matches the costs we currently externalize to the Pypi team.
> >
> > This suggestion seems uncontroversial to me. Thus I would like to start
> lazy
> > consensus. If there are no objections, I will assume lazy consensus on
> > stopping
> > nightly releases to Pypi in 72hrs.
> >
> > Best regards
> > Leonard
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message