mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kellen sunderland <kellen.sunderl...@gmail.com>
Subject Re: Testing examples in nightly build
Date Fri, 17 Aug 2018 07:54:08 GMT
@Vandana: Yes the backlog would need to be addressed, but I've been very
encouraged by your and cclauss's recent work in this area.  Thanks a ton
for the effort, I think it's really improving the quality.

On Thu, Aug 16, 2018 at 8:15 PM Marco de Abreu
<marco.g.abreu@googlemail.com.invalid> wrote:

> These are great ideas, sounds very good to me. Thanks for your efforts
> around the user experience!
>
> -Marco
>
> Vandana Kannan <vandana268@gmail.com> schrieb am Do., 16. Aug. 2018,
> 18:27:
>
> > @Marco: Most of the examples are standalone applications and not
> > notebooks. We could start off by enabling tests for examples that are
> > either notebooks or those that can be executed with command line options.
> >
> > Some of the problems that we may come across are with examples that load
> > large datasets (we could probably use a smaller dataset, but it depends
> on
> > the example), or examples that have a long training time (we could
> probably
> > execute them for 1 epoch). Straight Dope nightly tests cap the execution
> > time to 10 minutes per notebook. We could do the same here wherever
> > possible.
> >
> > @Kellen: Good idea to include flake8/pylint checks for examples in
> nightly
> > builds - we would need to fix the backlog (
> > https://github.com/apache/incubator-mxnet/issues/12205) before enabling.
> > Yes, any contribution from the community on any of these TODO items,
> would
> > be great.
> >
> > - Vandana
> >
> > On 2018/08/16 06:40:56, kellen sunderland <kellen.sunderland@gmail.com>
> > wrote:
> > > I think it would be very beneficial to start fleshing out the nighties.
> > > They provide a lot of value at a relatively small cost.  Any
> > contributions
> > > from the community would be appreciated.
> > >
> > > Things I could see being beneficial:
> > > * Long running tests
> > > * In depth flake8/pylint linters
> > > * cudamemcheck builds
> > > * Asan builds.
> > >
> > > On Thu, Aug 16, 2018, 12:24 AM Marco de Abreu
> > > <marco.g.abreu@googlemail.com.invalid> wrote:
> > >
> > > > Hello,
> > > >
> > > > I think this is a great idea! Thanks a lot for fixing all the
> problems
> > in
> > > > our examples.
> > > >
> > > > Do you see any problems that could come up if we just run them one by
> > one?
> > > > We already got a pipeline that allows us to verify jupyter notebooks,
> > but
> > > > afaik the examples are standalone files with a main function, right?
> > > >
> > > > Best regards,
> > > > Marco
> > > >
> > > > Vandana Kannan <vandana268@gmail.com> schrieb am Mi., 15. Aug. 2018,
> > > > 23:44:
> > > >
> > > > > Hi All,
> > > > >
> > > > > Recently we saw that there were quite a few Pylint
> undefined-variable
> > > > > errors in MXNet code, a majority of them in the example folder.
> These
> > > > > errors highlight that there are paths in the examples that are
> > broken and
> > > > > these have not been caught during testing.
> > > > >
> > > > > These errors are later reported as issues by users. For example,
> > > > > https://github.com/apache/incubator-mxnet/issues/11278.
> > > > >
> > > > > It would be a good idea to include tests for all examples in the
> > nightly
> > > > > builds (Similar to
> > > > > incubator-mxnet/tests/nightly/test_image_classification.sh). Side
> > note:
> > > > We
> > > > > might have to see what to do about examples that include special
> > setup
> > > > > instructions or take too long to execute.
> > > > >
> > > > > Any thoughts or suggestions on writing these tests, feasibility or
> > > > > previous attempts at including these tests, would be helpful.
> > > > >
> > > > > Thanks,
> > > > > Vandana
> > > > >
> > > >
> > >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message