mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yuan Tang <terrytangy...@gmail.com>
Subject Re: [Discuss] Experiment Reproducibility in MXNet
Date Tue, 03 Sep 2019 19:14:54 GMT
I don’t think we’d want to re-invent the wheel as there are many solutions
exist already. Another solution besides mlflow is Kubeflow Pipelines:
https://github.com/kubeflow/pipelines

On Mon, Sep 2, 2019 at 10:12 PM Naveen Swamy <mnnaveen@gmail.com> wrote:

> Look at https://mlflow.org/
>
> > On Sep 2, 2019, at 7:02 PM, Chaitanya Bapat <chai.bapat@gmail.com>
> wrote:
> >
> > Hello MXNet community,
> >
> > Reproducibility of ML experiments carried out by data scientists,
> analysts
> > and experts is the talk of the town.
> >
> > While listening to TWiML's latest podcast - Managing Deep Learning
> > Experiments with Lukas Biewald [1], he mentions the company Weights and
> > Biases [2] [3]
> >
> > Brief
> > - Reproducibility crisis in ML
> > - Let alone the latest research papers, even your own experiments (say
> from
> > 1 month ago) are not reproducible
> > - Solution :
> > 1. Versioning
> > Takes snapshots to store versions - Code, Data, Parameters and Hyper
> > parameters
> > Versioning or Snapshotting falls in the realm of data management. Notable
> > companies - DVC and Pachyderm.
> >
> > 2. Visualization
> > Builds on top of Tensorboard (TBoard). But solves its shortcomings
> > - Targeted for distributed training (unlike TBoard)
> > - Visualizes wrt several experiments (not just a single run)
> >
> > 3. Collaboration
> > Making this cloud based, allows cross-team collaboration.
> >
> > *MXNet*
> > From MXNet's point of view, we can discuss if it's worthwhile to have
> this
> > (many positives point towards a yes) and if so we can explore following
> > options -
> > a. Work with W&B for building support for using it with MXNet (currently
> > they have Tensorflow (TF) and PyTorch (PT) supported)
> > b. Build something in-house on similar lines that would involve
> significant
> > engineering effort, discussion.
> >
> > So I wanted to know what does the community think about this?
> >
> > Thanks,
> > Chai
> >
> > [1]
> >
> https://twimlai.com/twiml-talk-295-managing-deep-learning-experiments-with-lukas-biewald
> > [2] https://www.wandb.com
> > [3] https://github.com/wandb
> >
> > --
> > *Chaitanya Prakash Bapat*
> > *+1 (973) 953-6299*
> >
> > [image: https://www.linkedin.com//in/chaibapat25]
> > <https://github.com/ChaiBapchya>[image:
> https://www.facebook.com/chaibapat]
> > <https://www.facebook.com/chaibapchya>[image:
> > https://twitter.com/ChaiBapchya] <https://twitter.com/ChaiBapchya
> >[image:
> > https://www.linkedin.com//in/chaibapat25]
> > <https://www.linkedin.com//in/chaibapchya/>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message