mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pedro Larroy <pedro.larroy.li...@gmail.com>
Subject Re: Catching segmentation faults in nosetests
Date Thu, 21 Jun 2018 21:52:51 GMT
Correct. I don't think you are framing this correctly, as I don't recommend
you going into the rabbit hole of trying to recover from the segfault. Let
it crash and mark the full suite as failed.

The option is to execute each suite on separate invocations of nose /
python and then merge the xunit results in a final step marking the ones
that crashed as such.

You can wrap / instrument the executions of each suite via nose API and
then fork, or execute from above with a script to catch the segfaults for
every test or suite.


Pedro.

On Thu, Jun 21, 2018 at 2:23 PM Marco de Abreu
<marco.g.abreu@googlemail.com.invalid> wrote:

> Thank you Pedro. But this would still crash the nosetests process and thus
> prevent the summary from being created, right?
>
> On Thu, Jun 21, 2018 at 11:14 PM Pedro Larroy <
> pedro.larroy.lists@gmail.com>
> wrote:
>
> > A crash in the library is not an error in the test, is a different
> > situation.
> >
> > I suggest adding these flags to get stack traces and addressing the crash
> > as a separate bug.
> >
> >
> >
> https://github.com/larroy/mxnet/commit/46389e5851a60d06c37755e5772e6cbcd71b0080
> >
> > Check the return code when executing the test ($? variable in bash for
> > example) and it will have the value explained here (
> >
> >
> https://stackoverflow.com/questions/14599670/what-error-code-does-a-process-that-segfaults-return
> > )
> > set. Then you can mark the full suite as crashed.
> >
> > Pedro.
> >
> > On Thu, Jun 21, 2018 at 1:35 PM Marco de Abreu
> > <marco.g.abreu@googlemail.com.invalid> wrote:
> >
> > > Hello,
> > >
> > > is anybody aware of a way to catch segmentation faults as part of the
> > > nosetests execution and log them as ERROR? Right now, the nosetests
> > process
> > > gets terminated without a stack trace or further test execution. This
> has
> > > additional significance because our result-recording that has been
> > > introduced in [1] does not get executed if nosetests does not run until
> > the
> > > end. This means, we're not able to record and track any segmentation
> > > faults.
> > >
> > > If there is anybody in this community who has experience with catching
> > > segmentation faults without terminating the nosetests parent process,
> I'd
> > > really appreciate some guidance.
> > >
> > > Best regards,
> > > Marco
> > >
> > > [1]: https://github.com/apache/incubator-mxnet/pull/11199
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message