singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [singa] nudles commented on a change in pull request #587: SINGA-504 Add Gemm operator for autograd and onnx
Date Sat, 01 Feb 2020 12:57:07 GMT
nudles commented on a change in pull request #587: SINGA-504 Add Gemm operator for autograd
and onnx
URL: https://github.com/apache/singa/pull/587#discussion_r373778730
 
 

 ##########
 File path: python/singa/autograd.py
 ##########
 @@ -2760,3 +2760,107 @@ def backward(self, dy):
 
 def reciprocal(x):
     return Reciprocal()(x)[0]
+
+
+class Gemm(Operation):
+    def __init__(self, alpha=1.0, beta=1.0, transA=0, transB=0):
+        """
+        init a General Matrix multiplication(Gemm) operator
+        Compute Y = alpha * A' * B' + beta * C, where input tensor A has shape (M, K) or
(K, M), input tensor B has shape (K, N) or (N, K), input tensor C is broadcastable to shape
(M, N), and output tensor Y has shape (M, N).
+        A' = transpose(A) if transA else A
+        B' = transpose(B) if transB else B
+        Args:alpha: 
+            float, Scalar multiplier for the product of input tensors A * B.
+        Args:beta: 
+            float, Scalar multiplier for input tensor C.
+        Args:transA: 
+            int, Whether A should be transposed
+        Args:transB: 
+            int, Whether B should be transposed
+        Returns: 
+            tensor, the output
+        """
+        super(Gemm, self).__init__()
+        self.alpha = alpha
+        self.beta = beta
+        self.transA = transA
+        self.transB = transB
+
+    def forward(self, A, B, C=None):
+        """
+        forward propogation of Gemm
+        Args:A: 
+            tensor, The shape of A should be (M, K) if transA is 0, or (K, M) if transA is
non-zero.
+        Args:B: 
+            tensor, The shape of B should be (K, N) if transB is 0, or (N, K) if transB is
non-zero.
+        Args:C: 
+            tensor(optional), Optional input tensor C. If not specified, the computation
is done as if C is a scalar 0. The shape of C should be unidirectional broadcastable to (M,
N).
+        Returns: 
+            tensor, the output
+        """
+        _A = singa.DefaultTranspose(A) if self.transA == 1 else A
+        _B = singa.DefaultTranspose(B) if self.transB == 1 else B
+        if training:
+            self.inputs = (_A, _B, C)
+        tmpM = singa.MultFloat(singa.Mult(_A, _B), self.alpha)
 
 Review comment:
   how to optimize it?
   I think MultiWithScale is more efficient as it calls the cpp/cuda gemm directly instead
of doing the matrix multiplication and addition separately..

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message