mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gautam <gautamn...@gmail.com>
Subject Re: CI problems
Date Wed, 27 Sep 2017 18:12:33 GMT
Hi Chris,

    There could be possibility that someone might have merged the changes
without having all checks running, or overlooked the result. Currently we
don't have any mechanism where git farm can reject such merge where CI/Unit
test fails. I will try to explore any such possibilities. Meanwhile I would
rather disable those fail tests with a git hub issue so that build can be
clean for now. We can revisit those issue later.




On Wed, Sep 27, 2017 at 11:02 AM, kellen sunderland <
kellen.sunderland@gmail.com> wrote:

> I share in your frustration Chris. I've also spent a fair amount of time in
> the past few days digging through console logs to try and see if there was
> anything actionable.  I haven't noticed any tests that were failing
> consistently, maybe you can post an issue with some specific tests?  For me
> the larger issue is Sanity Check failures and segfaults at the end of test
> runs (after all tests pass).  I'm assuming that people are working on the
> Sanity Check issues.  If there's anything external contributors can do
> please let us know.
>
> On Wed, Sep 27, 2017 at 5:48 PM, Chris Olivier <cjolivier01@gmail.com>
> wrote:
>
> > By the way, I am not referring to a few tests that are known to fail
> 1%-10%
> > or so of the time (ie test_batchnorm_training) and are being actively
> > worked on. I am referring to tests that fail 100% of the time and are
> still
> > merged into master, and thus propagate to all branches when sync'd from
> > master.
> >
> > On Wed, Sep 27, 2017 at 8:43 AM, Chris Olivier <cjolivier01@gmail.com>
> > wrote:
> >
> > > How are so many broken unit tests getting into master?  Is stuff being
> > > merged without passing CI/unit testing?  I have been trying to get
> three
> > > PR's to build for over a week now.  Each time it's some broken test or
> > > another that has nothing to do with my code changes.  It's extremely
> > > frustrating -- I waste whole days on this, trying to figure out why my
> > code
> > > is breaking strange things only to realize later it's broken in all
> > > branches.
> > >
> >
>



-- 
Best Regards,
Gautam Kumar

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message