mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco de Abreu <marco.g.ab...@gmail.com>
Subject Re: Stopping nightly releases to Pypi
Date Tue, 03 Dec 2019 18:22:22 GMT
Isn't there an s3 endpoint in Beijing?

It seems like this topic still warrants some discussion and thus I'd prefer
if we don't move forward with lazy consensus.

-Marco

Tao Lv <mutouorz@gmail.com> schrieb am Di., 3. Dez. 2019, 14:31:

> * For pypi, we can use mirrors.
>
> On Tue, Dec 3, 2019 at 9:28 PM Tao Lv <mutouorz@gmail.com> wrote:
>
> > As we have many users in China, I'm considering the accessibility of S3.
> > For pip, we can mirrors.
> >
> > On Tue, Dec 3, 2019 at 3:24 PM Lausen, Leonard <lausen@amazon.com.invalid
> >
> > wrote:
> >
> >> I would like to remind everyone that lazy consensus is assumed if no
> >> objections
> >> are raised before 2019-12-05 at 05:42 UTC. There has been some
> discussion
> >> about
> >> the proposal, but to my understanding no objections were raised.
> >>
> >> If the proposal is accepted, MXNet releases would be installed via
> >>
> >>    pip install mxnet
> >>
> >> And release candidates via
> >>
> >>   pip install --pre mxnet
> >>
> >> (or with the respective cuda version specifier appended etc.)
> >>
> >> To obtain releases built automatically from the master branch, users
> >> would need
> >> to specify something like "-f
> >> http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html" option to pip.
> >>
> >> Best regards
> >> Leonard
> >>
> >> On Mon, 2019-12-02 at 05:42 +0000, Lausen, Leonard wrote:
> >> > Hi MXNet Community,
> >> >
> >> > since more than 2 months our binary Python nightly releases published
> >> on Pypi
> >> > are broken. The problem is that our binaries exceed Pypi's size limit.
> >> > Decreasing the binary size by adding compression breaks third-party
> >> libraries
> >> > loading libmxnet.so
> >> https://github.com/apache/incubator-mxnet/issues/16193
> >> >
> >> > Sheng requested Pypi to increase their size limit:
> >> > https://github.com/pypa/pypi-support/issues/50
> >> >
> >> > Currently "the biggest cost for PyPI from [the many MXNet binaries
> with
> >> > nightly
> >> > release to Pypi] is the bandwidth consumed when several hundred
> mirrors
> >> > attempt
> >> > to mirror each release immediately after it's published". So Pypi is
> not
> >> > inclined to allow us to upload even larger binaries on a nightly
> >> schedule.
> >> > Their compromise is to allow it on a weekly cadence.
> >> >
> >> > However, I would like the community to revisit the necessity of
> >> releasing pre-
> >> > release binaries to Pypi on a nightly (or weekly) cadence. Instead, we
> >> can
> >> > release nightly binaries ONLY to a public S3 bucket and instruct users
> >> to
> >> > install from there. On our side, we only need to prepare a html
> >> document that
> >> > contains links to all released nightly binaries.
> >> > Finally users will install the nightly releases via
> >> >
> >> >   pip install --pre mxnet-cu101 -f
> >> http://mxnet.s3.amazonaws.com/mxnet-cu101/
> >> > nightly.html
> >> >
> >> > Instead of
> >> >
> >> >   pip install --pre mxnet-cu101
> >> >
> >> > Of course proper releases and release candidates should still be made
> >> > available
> >> > via Pypi. Thus releases would be installed via
> >> >
> >> >   pip install mxnet-cu101
> >> >
> >> > And release candidates via
> >> >
> >> >   pip install --pre mxnet-cu101
> >> >
> >> > This will substantially reduce the costs of the Pypi project and in
> fact
> >> > matches
> >> > the installation experience provided by PyTorch. I don't think the
> >> benefit of
> >> > not including "-f
> >> http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html"
> >> > matches the costs we currently externalize to the Pypi team.
> >> >
> >> > This suggestion seems uncontroversial to me. Thus I would like to
> start
> >> lazy
> >> > consensus. If there are no objections, I will assume lazy consensus on
> >> > stopping
> >> > nightly releases to Pypi in 72hrs.
> >> >
> >> > Best regards
> >> > Leonard
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message