mxnet-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] hetong007 commented on issue #11019: Gluon SoftmaxCELoss does not converge, for large number of classes.
Date Thu, 31 May 2018 19:03:54 GMT
hetong007 commented on issue #11019: Gluon SoftmaxCELoss does not converge, for large number
of classes.
URL: https://github.com/apache/incubator-mxnet/issues/11019#issuecomment-393642796
 
 
   When trying to reproduce, I got the following error message:
   
   ```
   INFO:root:Epoch[0] Batch [1940]	Speed: 1124.429415 samples/sec	acc=0.000416
   INFO:root:Epoch[0] Batch [1960]	Speed: 1130.981356 samples/sec	acc=0.000412
   INFO:root:Epoch[0] Batch [1980]	Speed: 1127.127662 samples/sec	acc=0.000408
   lr-batch-epoch: 0.1 2000
   testing verification..
   Traceback (most recent call last):
     File "train.py", line 746, in <module>
       main()
     File "train.py", line 743, in main
       train_net(args)
     File "train.py", line 660, in train_net
       _batch_callback()
     File "train.py", line 594, in _batch_callback
       acc_list = ver_test(mbatch)
     File "train.py", line 453, in ver_test
       acc1, std1, acc2, std2, xnorm, embeddings_list = verification.test(ver_list[i], net,
ctx, batch_size = args.batch_size)
     File "/home/ubuntu/insightface/gluon/verification.py", line 269, in test
       embeddings = sklearn.preprocessing.normalize(embeddings)
     File "/usr/local/lib/python2.7/dist-packages/sklearn/preprocessing/data.py", line 1412,
in normalize
       estimator='the normalize function', dtype=FLOAT_DTYPES)
     File "/usr/local/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 453,
in check_array
       _assert_all_finite(array)
     File "/usr/local/lib/python2.7/dist-packages/sklearn/utils/validation.py", line 44, in
_assert_all_finite
       " or a value too large for %r." % X.dtype)
   ValueError: Input contains NaN, infinity or a value too large for dtype('float64').
   ```
   
   Is it something to be expected?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message