mxnet-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco de Abreu <marco.g.ab...@googlemail.com>
Subject Problems with test_sparse_operator.test_sparse_mathematical_core
Date Wed, 09 May 2018 12:32:15 GMT
Hello,

I'm currently working on auto scaling and encountering a consistent test
failure on CPU. At the moment, I'm not really sure what's causing this,
considering the setup should be identical.

http://jenkins.mxnet-ci-dev.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/ci-master/557/pipeline/694

======================================================================

FAIL: test_sparse_operator.test_sparse_mathematical_core

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/usr/local/lib/python3.5/dist-packages/nose/case.py", line 198, in
runTest

    self.test(*self.arg)

  File "/work/mxnet/tests/python/unittest/common.py", line 157, in test_new

    orig_test(*args, **kwargs)

  File "/work/mxnet/tests/python/unittest/test_sparse_operator.py", line
1084, in test_sparse_mathematical_core

    density=density, ograd_density=ograd_density)

  File "/work/mxnet/tests/python/unittest/test_sparse_operator.py", line
1056, in check_mathematical_core

    density=density, ograd_density=ograd_density)

  File "/work/mxnet/tests/python/unittest/test_sparse_operator.py", line
698, in check_sparse_mathematical_core

    assert_almost_equal(arr_grad, input_grad, equal_nan=True)

  File "/work/mxnet/python/mxnet/test_utils.py", line 493, in
assert_almost_equal

    raise AssertionError(msg)

AssertionError:

Items are not equal:

Error nan exceeds tolerance rtol=0.000010, atol=0.000000.  Location of
maximum error:(0, 0), a=inf, b=-inf

 a: array([[inf],

       [inf],

       [inf],...

 b: array([[-inf],

       [-inf],

       [-inf],...

-------------------- >> begin captured stdout << ---------------------

pass 0

0.0, 0.0, False

--------------------- >> end captured stdout << ----------------------

-------------------- >> begin captured logging << --------------------

common: INFO: Setting test np/mx/python random seeds, use
MXNET_TEST_SEED=2103230797 to reproduce.

--------------------- >> end captured logging << ---------------------


Does this ring any bells?

Thanks in advance!

-Marco

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message