From dev-return-4133-archive-asf-public=cust-asf.ponee.io@singa.apache.org Tue Feb 4 11:46:07 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4358818064E for ; Tue, 4 Feb 2020 12:46:07 +0100 (CET) Received: (qmail 57980 invoked by uid 500); 4 Feb 2020 11:46:06 -0000 Mailing-List: contact dev-help@singa.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@singa.apache.org Delivered-To: mailing list dev@singa.apache.org Received: (qmail 57970 invoked by uid 99); 4 Feb 2020 11:46:06 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Feb 2020 11:46:06 +0000 From: GitBox To: dev@singa.apache.org Subject: [GitHub] [singa] joddiy commented on a change in pull request #587: SINGA-504 Add Gemm operator for autograd and onnx Message-ID: <158081676654.341.1753135685727672091.gitbox@gitbox.apache.org> References: In-Reply-To: Date: Tue, 04 Feb 2020 11:46:06 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit joddiy commented on a change in pull request #587: SINGA-504 Add Gemm operator for autograd and onnx URL: https://github.com/apache/singa/pull/587#discussion_r374623856 ########## File path: python/singa/autograd.py ########## @@ -2760,3 +2760,107 @@ def backward(self, dy): def reciprocal(x): return Reciprocal()(x)[0] + + +class Gemm(Operation): + def __init__(self, alpha=1.0, beta=1.0, transA=0, transB=0): + """ + init a General Matrix multiplication(Gemm) operator + Compute Y = alpha * A' * B' + beta * C, where input tensor A has shape (M, K) or (K, M), input tensor B has shape (K, N) or (N, K), input tensor C is broadcastable to shape (M, N), and output tensor Y has shape (M, N). + A' = transpose(A) if transA else A + B' = transpose(B) if transB else B + Args:alpha: + float, Scalar multiplier for the product of input tensors A * B. + Args:beta: + float, Scalar multiplier for input tensor C. + Args:transA: + int, Whether A should be transposed + Args:transB: + int, Whether B should be transposed + Returns: + tensor, the output + """ + super(Gemm, self).__init__() + self.alpha = alpha + self.beta = beta + self.transA = transA + self.transB = transB + + def forward(self, A, B, C=None): + """ + forward propogation of Gemm + Args:A: + tensor, The shape of A should be (M, K) if transA is 0, or (K, M) if transA is non-zero. + Args:B: + tensor, The shape of B should be (K, N) if transB is 0, or (N, K) if transB is non-zero. + Args:C: + tensor(optional), Optional input tensor C. If not specified, the computation is done as if C is a scalar 0. The shape of C should be unidirectional broadcastable to (M, N). + Returns: + tensor, the output + """ + _A = singa.DefaultTranspose(A) if self.transA == 1 else A + _B = singa.DefaultTranspose(B) if self.transB == 1 else B + if training: + self.inputs = (_A, _B, C) + tmpM = singa.MultFloat(singa.Mult(_A, _B), self.alpha) Review comment: I just try this MultiWithScale, but it cannot support the Unidirectional Broadcasting for bias. https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org With regards, Apache Git Services