singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Yeung (Jira)" <j...@apache.org>
Subject [jira] [Created] (SINGA-502) Avoid moving data between host and gpu devices in some operations
Date Tue, 21 Jan 2020 08:40:00 GMT
Chris Yeung created SINGA-502:
---------------------------------

             Summary: Avoid moving data between host and gpu devices in some operations
                 Key: SINGA-502
                 URL: https://issues.apache.org/jira/browse/SINGA-502
             Project: Singa
          Issue Type: Improvement
          Components: Core
            Reporter: Chris Yeung


Some functions move data between GPU and host memory, which should be fixed for many reasons
such as efficiency and asynchronization (and buffering operation in the future). For example:

The softmax_cross_entropy move to data to host and then back to gpu, so the whole function
needed to be changed:

class SoftMaxCrossEntropy(Operation):   

def __init__(self, t):       
  super(SoftMaxCrossEntropy, self).__init__()       
  self.t = t.data

def forward(self, x):       
  self.p = singa.SoftMax(x)       
  loss = CTensor((1,), self.p.device())       
  ret = singa.CrossEntropyFwd(self.p, self.t)       
  loss.SetFloatValue(singa.SumAsFloat(ret) / x.shape()[0])       
  return loss

Here the SumAsFloat return a c++ float value,  and this value is read back to gpu in the SetFloatValue
function. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message