chrishkchris edited a comment on issue #488: SINGA -475 add Sign operator to singa
URL: https://github.com/apache/incubator-singa/pull/488#issuecomment-519856801
In my opinion:
For the function y=sign(x) , y = 1 when x > 0, y = -1 when x < 0. Therefore, the derivative is 0 except x = 0.
The function y=sign(x) is discontinuous at x=0 where the derivative is undefined in the mathematical logic. However, an undefined derivative leads to a nan number if we use it for backpropagation, so instead we used a derivative of zero also at x=0.
As a result, dx is always 0 while the output shape (size of array) is the same as input shape.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services