singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [singa] chrishkchris commented on issue #674: Autograd Layer constructor
Date Sun, 12 Apr 2020 06:23:27 GMT
chrishkchris commented on issue #674: Autograd Layer constructor
URL: https://github.com/apache/singa/issues/674#issuecomment-612570634
 
 
   > > > I think it's not an issue. When we use the computational graph, initialization
operations won't be buffered since we just need to execute them once. For these operations,
I just execute them immediately instead of buffering them into the graph at present. So before
running the entire graph, all parameter tensors will be initialized.
   > > 
   > > 
   > > but you need to identify the operations for initialization.. how to tell if an
operation is for initialization or not?
   > 
   > At present, if the operations are for initialization, they are likely to be in the
constructor function. So I just turn on the graph before calling the __call__ function of
Layer and forward and backward function of Operation. So all the initialization operations
in the constructor will be executed immediately. For those initialization operations in the
functions I mentioned above, I turn off the buffer before calling them. Like this, [turn off
graph](https://github.com/apache/singa/blob/master/python/singa/opt.py#L155).
   
   @XJDKC Yes, that' why prof. suggest add the wrapper to it:
   ```
   I think it would be better to define a new function to wrap these lines:
   def execute_once(fn):
       get flag
       disable graph
       fn()
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message