singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-singa] dcslin commented on issue #540: added softmax with axis
Date Tue, 08 Oct 2019 17:51:29 GMT
dcslin commented on issue #540: added softmax with axis
URL: https://github.com/apache/incubator-singa/pull/540#issuecomment-539628035
 
 
   > > > still has some problems, the output of multiple dimension inputs is not
correct.
   > > > please check:
   > > > ```
   > > > x_0 = np.array([[0, 1, 2, 3], [10000, 10001, 10002, 10003]]).astype(np.float32)
   > > > # axis is 1
   > > > # expected output [[0.0320586, 0.08714432, 0.23688284, 0.64391428],
   > > > #                 [0.0320586, 0.08714432, 0.23688284, 0.64391428]]
   > > > ```
   > > 
   > > 
   > > Hi @joddiy , this is updated, could you please help to review?
   > 
   > Hi shicong, the axis and output are almost correct. However, the result will overflow
if the input values are too big. For example, for input of [0, 1, 2, 3], the result is correct.
But for input of 10000, 10001, 10002, 10003], the result will be [nan, nan, nan, nan].
   > 
   > Please use this formation:
   > 
   > ```
   > def softmax(x):
   >     """Compute softmax values for each sets of scores in x."""
   >     e_x = np.exp(x - np.max(x))
   >     return e_x / e_x.sum()
   > ```
   > 
   > **not directly use x as input, instead subtracting the max value of x from x to avoid
the overflow.**
   
   Hi @joddiy , thank you for the comment. I am still looking into this issue, as `max()`
is not currently implemented in backend yet.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message