singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [singa] chrishkchris commented on issue #674: Autograd Layer constructor
Date Sun, 12 Apr 2020 01:58:33 GMT
chrishkchris commented on issue #674: Autograd Layer constructor
URL: https://github.com/apache/singa/issues/674#issuecomment-612550025
 
 
   > > I think it's not an issue. When we use the computational graph, initialization
operations won't be buffered since we just need to execute them once. For these operations,
I just execute them immediately instead of buffering them into the graph at present. So before
running the entire graph, all parameter tensors will be initialized.
   > 
   > but you need to identify the operations for initialization.. how to tell if an operation
is for initialization or not?
   
   If the initization statement is in forward function: to tell that it is an initialization
operation not for buffering, we need to add three lines:
   1. Before the initialization statament not for buffering, add two lines:
   flag = param.device.graph_enabled()
   param.device.EnableGraph(False)
   2. After the statement, add one line
   param.device.EnableGraph(flag)
   Note that param is any input tensor that has the attribute "device" for us to use
    

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message