singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [singa] nudles commented on issue #696: Refactor autograd module
Date Thu, 14 May 2020 03:29:15 GMT

nudles commented on issue #696:
URL: https://github.com/apache/singa/issues/696#issuecomment-628364838


   > 
   > 
   > > 1. Before we call Module.forward(), we can randomly fill the placeholder tensors.
   > > 2. We can make Layer.init() optional. To implement a new layer, the parameter
initialization can be done within the `__call__` method or in a `init()` method. It is up
to the contributor.
   > > 
   > > Any comments on the drawbacks?
   > > @dcslin @XJDKC
   > 
   >     1. If we separate the initialization and the forward propagation, there is no
need to fill the placeholder. The data of tensors will only be accessed in the forward propagation.
For initialization, we just access their shapes, types and so on.
   Then we need a method like `infer_output_shapes(self, input_shapes)`; otherwise, we have
to call the forward method to get the output shapes from the output tensor. I prefer to call
the forward function to avoid adding a new method to each layer.  
   > 
   >     2. That's great. But if users move the initialization into __call__ function,
they should determine whether the layer has been initialized by themselves.
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message