mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco de Abreu <marco.g.ab...@gmail.com>
Subject Re: demise of dmlc.ml domain & new Julia docs hosting
Date Thu, 18 Jul 2019 21:36:22 GMT
Great, thanks for the links and for working on the plan. I think the
community should align on the requirements of the new website as long as
it's still in development. We can then do an analysis in how far the
current state of the beta website differs from the requirements and make
that part of the transition plan.

Would you mind leading the technical requirement gathering?

Best regards,
Marco

On Thu, Jul 18, 2019 at 11:22 PM Aaron Markham <aaron.s.markham@gmail.com>
wrote:

> What I have written doesn't directly align with the beta site. The site was
> created by Mu, and several people have contributed to updating it with
> content. The repo is here  https://github.com/mli/new-docs
> There's no architecture doc afaik.
>
> I have docs for a redesign and new ux in the wiki:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103089084
> I've tried to apply some of the IA to the beta site, but I'm constrained by
> the Sphinx template.
>
> There's decent progress in the beta site, but it has the drawback of
> lock-in with Sphinx again. And covers only Python and R.
>
> I will be writing a doc to cover a transition plan. I will try to layer it,
> so we have the bare minimum all the way to the optimal route. That being
> said, any plan requires resources. It's not going to be easy.
>
> On Thu, Jul 18, 2019, 14:01 Marco de Abreu <marco.g.abreu@gmail.com>
> wrote:
>
> > I see. Would you mind linking the thread/doc where we discussed the
> > requirements for the new website?
> >
> > -Marco
> >
> > On Thu, Jul 18, 2019 at 9:49 PM Aaron Markham <aaron.s.markham@gmail.com
> >
> > wrote:
> >
> > > I agree with you. I'd like the website to be fairly standard Jekyll and
> > > normal front end stuff, rather than a Sphinx template. Docs would be
> docs
> > > built by doxygen, Sphinx, Scala, java, etc, based on their native docs
> > > generation, and we'd get a microsite for each language binding. If one
> > > breaks it doesn't break the whole website. You can build them
> separately
> > or
> > > build a particular version. I made the settings system accomplish that
> > with
> > > our Sphinx build, but the template we have tied with the old versions
> > keeps
> > > us locked into a complex build.
> > >
> > > If we must support old versions then this notion needs to be part of
> the
> > > new beta Python site, and as far as I know, that's not planned. It is
> not
> > > supported in the beta site now.
> > >
> > > On Thu, Jul 18, 2019, 12:31 Marco de Abreu <marco.g.abreu@gmail.com>
> > > wrote:
> > >
> > > > I'm not entirely sure about dropping docs for old versions to be
> > honest.
> > > > The root cause in my opinion is the fact that the build is waaaay to
> > > > complicated. If it would have less dependencies and be more
> > > > straight-forward, it should be a matter of seconds. The key here
> could
> > be
> > > > to separate the docs from the website. Doc generation should be a
> > simple
> > > > matter of "run doxygen" (or whatever we use). Right now, the website
> > > build
> > > > process intermingles every aspect of it. If each version (and
> language)
> > > > would just output a docs folder that has no outgoing references, they
> > > could
> > > > be created entirely independent of another. Right now, we regenerate
> > > every
> > > > version for every single build - that's not necessary.
> > > >
> > > > Considering how many people are unable to use the latest version of
> > > MXNet,
> > > > I'd prefer if we provide the docs as long as the release binaries are
> > > > available.
> > > >
> > > > -Marco
> > > >
> > > > Aaron Markham <aaron.s.markham@gmail.com> schrieb am Do., 18. Juli
> > 2019,
> > > > 21:20:
> > > >
> > > > > I'd have it check signatures. Ssl is optional I think. China users
> > seem
> > > > to
> > > > > have issues with SSL... That may have changed, but I think we might
> > > have
> > > > to
> > > > > work around that for background downloads for models.
> > > > >
> > > > > With regard to the current links to dmlc.ml, I asked ivy to just
> get
> > > rid
> > > > > of
> > > > > them with # if necessary. Better to 404 than go to malware. This
> > > requires
> > > > > going back into previous release branches and purging those
> > references.
> > > > >
> > > > > The website is publishing regularly again. I had trouble getting
> > Julia
> > > to
> > > > > work in CI for generating the Julia docs, but I have it working
> now.
> > > > >
> > > > > As we move forward with yet another release, plus the potential
> > launch
> > > of
> > > > > the new Python docs site we need to consider reducing complexity
of
> > the
> > > > > build and prune old versions. I think we should only host master
or
> > the
> > > > > most recent release, and provide instructions on how to generate
> > other
> > > > > versions. Otherwise the transition will be too difficult and
> > > disruptive.
> > > > >
> > > > > On Tue, Jul 16, 2019, 16:41 Marco de Abreu <
> marco.g.abreu@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > While we are at it, we should maybe also take about SSL to avoid
> > > these
> > > > > > kinds of downloads in future.
> > > > > >
> > > > > > -Marco
> > > > > >
> > > > > > Aaron Markham <aaron.s.markham@gmail.com> schrieb am Mi.,
17.
> Juli
> > > > 2019,
> > > > > > 01:20:
> > > > > >
> > > > > > > Hi Hen,
> > > > > > > We still have all of the these models that are referenced
in
> this
> > > > > > > issue: https://github.com/apache/incubator-mxnet/issues/15410
> > > > > > > I think it's a pretty big security breach to have systems
that
> do
> > > > > > > automatic downloading point to a malware site. Anything
that
> > > fetches
> > > > > > > binary data should be well secured and it really lowers
trust
> if
> > > > > > > assets send customers to malware sites.
> > > > > > > There might be more.
> > > > > > >
> > > > > > > On Sat, Jul 13, 2019 at 1:00 PM Hen <bayard@apache.org>
wrote:
> > > > > > > >
> > > > > > > > Nice work :)
> > > > > > > >
> > > > > > > > What other links does the project have that are outside
> Apache
> > > > Infra
> > > > > > > > control and have this risk?
> > > > > > > >
> > > > > > > > Outside of github.com/dmlc which is well known and
iiuc in
> > > process
> > > > > for
> > > > > > > > resolving.
> > > > > > > >
> > > > > > > > Hen
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Jul 9, 2019 at 4:46 PM Aaron Markham <
> > > > > > aaron.s.markham@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > The PR has passed CI. Please take a look.
> > > > > > > > > https://github.com/apache/incubator-mxnet/pull/15454
> > > > > > > > > We really need to get rid of those malware links,
and this
> > does
> > > > > that,
> > > > > > > > > restores the Julia docs with a local build, has
CI
> coverage,
> > > and
> > > > a
> > > > > > new
> > > > > > > > > Ubuntu guide (since I had to figure out how to
use Julia
> and
> > > > found
> > > > > > our
> > > > > > > > > docs for that were kind of broken).
> > > > > > > > > Cheers,
> > > > > > > > > Aaron
> > > > > > > > >
> > > > > > > > > On Thu, Jul 4, 2019 at 8:09 PM Iblis Lin <
> > iblis@hs.ntnu.edu.tw
> > > >
> > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > I will add +1 to the micro-site approach.
> > > > > > > > > > Since I have tried to take the generated
MD outputs from
> > > > > > > > > > Julia's doc system, but the syntax is incompatible
with
> > > > > > > > > > the MD plugin of Sphinx.
> > > > > > > > > >
> > > > > > > > > > Iblis Lin
> > > > > > > > > > 林峻頤
> > > > > > > > > >
> > > > > > > > > > On 7/4/19 12:12 PM, Aaron Markham wrote:
> > > > > > > > > > > Hi dev@,
> > > > > > > > > > > In case you missed the issues with
the dmlc.ml domain,
> > it
> > > > was
> > > > > > let
> > > > > > > go
> > > > > > > > > > > or sniped and now goes to a malware
site. [1] Several
> > > assets
> > > > > like
> > > > > > > > > > > models and the Julia documentation
were hosted there.
> > > > > > > > > > >
> > > > > > > > > > > I made some progress getting the Julia
docs generated
> as
> > > part
> > > > > of
> > > > > > > the
> > > > > > > > > > > regular website build flow. [2] It'll
work as long as
> you
> > > > have
> > > > > > > Julia
> > > > > > > > > > > installed and configured with MXNet.
I don't imagine
> you
> > > can
> > > > > use
> > > > > > > it to
> > > > > > > > > > > build the website natively on your
mac or windows box
> at
> > > this
> > > > > > point
> > > > > > > > > > > because all of the CI and related instructions
appear
> to
> > be
> > > > > only
> > > > > > > for
> > > > > > > > > > > Ubuntu. That being said, I added an
option to
> dev_menu.py
> > > so
> > > > > you
> > > > > > > can
> > > > > > > > > > > build the Julia docs with docker on
whatever host
> you're
> > > on.
> > > > > > > > > > >
> > > > > > > > > > > I tried to bring in the markdown files
and have the
> > website
> > > > > theme
> > > > > > > > > > > applied to them. Many things were then
broken - in
> large
> > > part
> > > > > due
> > > > > > > to
> > > > > > > > > > > some bugs in the website code related
to post
> processing
> > > and
> > > > > > > injection
> > > > > > > > > > > of dom elements. This would require
a rewrite of the
> > Julia
> > > > docs
> > > > > > to
> > > > > > > > > > > workaround the existing website bugs.
Rather than do
> > this,
> > > I
> > > > > just
> > > > > > > took
> > > > > > > > > > > the Julia site output, which has its
own look and feel
> > and
> > > > > nested
> > > > > > > in
> > > > > > > > > > > Julia's API directory. This is much
like how the scala
> > docs
> > > > and
> > > > > > the
> > > > > > > > > > > java docs are - using their own look
and feel as a
> > > > micro-site.
> > > > > I
> > > > > > > feel
> > > > > > > > > > > that this is the better approach for
now.
> > > > > > > > > > >
> > > > > > > > > > > The PR for the Julia docs is here:
> > > > > > > > > > > https://github.com/apache/incubator-mxnet/pull/15454
> > > > > > > > > > >
> > > > > > > > > > > Ivy and I created several new issues
to cover the
> broken
> > > > links
> > > > > > that
> > > > > > > > > > > will need fixing. Models that need
to be recovered or
> > > > recreated
> > > > > > and
> > > > > > > > > > > uploaded to a new location. I have
an s3 bucket that
> I've
> > > > been
> > > > > > > using
> > > > > > > > > > > for some public assets like this, but
I can't make
> > progress
> > > > on
> > > > > > > fixing
> > > > > > > > > > > those links when the models just don't
exist. And I
> don't
> > > > have
> > > > > > the
> > > > > > > > > > > bandwidth to regenerate the model zoo
and validate the
> > > > models.
> > > > > > > > > > >
> > > > > > > > > > > Any thoughts or suggestions are appreciated.
> > > > > > > > > > > Happy 4th of July!
> > > > > > > > > > >
> > > > > > > > > > > [1]
> > https://github.com/apache/incubator-mxnet/issues/15410
> > > > > > > > > > > [2]
> https://github.com/apache/incubator-mxnet/pull/15454
> > > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message