From dev-return-3439-archive-asf-public=cust-asf.ponee.io@singa.incubator.apache.org Sat Aug 17 14:53:22 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 1749C180181 for ; Sat, 17 Aug 2019 16:53:21 +0200 (CEST) Received: (qmail 42373 invoked by uid 500); 17 Aug 2019 14:53:21 -0000 Mailing-List: contact dev-help@singa.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@singa.incubator.apache.org Delivered-To: mailing list dev@singa.incubator.apache.org Received: (qmail 42363 invoked by uid 99); 17 Aug 2019 14:53:21 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 17 Aug 2019 14:53:21 +0000 From: GitBox To: dev@singa.apache.org Subject: [GitHub] [incubator-singa] chrishkchris commented on a change in pull request #515: SINGA-474 Add gemm operator Message-ID: <156605360138.3512.17237380570097076526.gitbox@gitbox.apache.org> Date: Sat, 17 Aug 2019 14:53:21 -0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit chrishkchris commented on a change in pull request #515: SINGA-474 Add gemm operator URL: https://github.com/apache/incubator-singa/pull/515#discussion_r314934800 ########## File path: python/singa/autograd.py ########## @@ -2316,3 +2315,37 @@ def backward(self, dy): def max(a,b): return Max()(a,b)[0] +class GEMM(Operation): + def __init__(self, alpha, beta, transA, transB): + self.alpha = alpha + self.beta = beta + self.transA = transA + self.transB = transB + super(GEMM, self).__init__() + + def forward(self, A, B, C): + if self.transA: + A = singa.DefaultTranspose(A) + if self.transB: + B = singa.DefaultTranspose(B) + if training: + self.A = A + self.B = B + + singa.MultWithScale(self.alpha, A, B, self.beta, C) + return C + + def backward(self, dY): + dC = singa.MultFloat(dY, self.beta) Review comment: Is the backward the same when transA and transB =1 or 0? Or it need to add transpose at the end, something like this? ```python def backward(self, dY): dC = singa.MultFloat(dY, self.beta) tB = singa.DefaultTranspose(self.B) tA = singa.DefaultTranspose(self.A) dA = singa.Mult(dY, tB) dB = singa.Mult(tA, dY) dA = singa.MultFloat(dA, self.alpha) dB = singa.MultFloat(dB, self.alpha) if self.transA: dA = singa.DefaultTranspose(dA) if self.transB: dB = singa.DefaultTranspose(dB) del self.A del self.B return dA, dB, dC ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org With regards, Apache Git Services