singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [singa] chrishkchris commented on issue #674: Autograd Layer constructor
Date Sun, 12 Apr 2020 06:20:02 GMT
chrishkchris commented on issue #674: Autograd Layer constructor
URL: https://github.com/apache/singa/issues/674#issuecomment-612570401
 
 
   > > > I think the Module class is quite similar to Layer, we can think about combining
them together. At present, users can't get params from the Module, they can only get them
from the Layer. If the Layer class is the subclass of Module, this problem will be solved.
   > > 
   > > 
   > > Yes, when we buffer the ops, there won't be any actual run, but all the parameters
size will be obtained after the call function.
   > 
   > > > > > I think the Module class is quite similar to Layer, we can think
about combining them together. At present, users can't get params from the Module, they can
only get them from the Layer. If the Layer class is the subclass of Module, this problem will
be solved.
   > > > > 
   > > > > 
   > > > > Yes, when we buffer the ops, there won't be any actual run, but all
the parameters size will be obtained after the call function.
   > > > 
   > > > 
   > > > After calling the constructor of Layer class.
   > > 
   > > 
   > > Constructor might not be enough, because in the case of RNN the CudnnRNNHandle.weight_size
is inferred from the input size. So this can only be obtained from the forward function. FYI,
this is the new Shicong's code in his PR (I guess this issue is for solving this PR problem):
   > > https://github.com/apache/singa/blob/19f2020fda36032fdc26dc5910dacbfcff638e7f/python/singa/autograd.py#L3406
   > 
   > Sorry, I haven't checked the newest code. Handles in the past code like ConvHandle
are initialized in the constructor, so I thought CudnnRNNHandle was the same.
   
   @XJDKC FYI Even ConvHandle is not initialized in the constructor of Layer but in the call
function, but the RNN is different from conv2d just because there is tensor initization in
the call function  https://github.com/apache/singa/blob/master/python/singa/autograd.py#L1758

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message