singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [singa] nudles commented on pull request #697: New Model Layer Operator API
Date Sun, 24 May 2020 02:10:14 GMT

nudles commented on pull request #697:
URL: https://github.com/apache/singa/pull/697#issuecomment-633166301


   > 
   > 
   > Now, users can define a new layer like this. (Just three functions)
   > 
   > ```
   > class MyLayer(layer.Layer):
   >     def __init__(self, a, b, c):
   >         super(MyLayer, self).__init__()
   >         self.l1 = layer.Linear(a,b)
   
   Using the new API, Linear layer can accept only one argument, i.e., the output feature
dim.
   
   >         self.l2 = layer.Linear(b,c)
   >         # just need to specify names of params and states
   >         self.param_names = ['W', 'b']
   
   there is no params explicitly defined by MyModel; therefore there should be no direct params
of MyModel. All params are from the sublayers of MyModel, i.e., l1 and l2.
   
   >         self.state_names = self.param_names + ['running_mean', 'running_var']
   
   I suggest to let the layers with params and states to override the set_param/get_param,
set/get_states functions.
   
   > 
   >     def initialize(self, x):  
   >         ## init params and states no need to enable the graph manually
   >         ## the graph will be enabled automatically
   >         pass
   > 
   >     def forward(self, x):
   >         y = self.l1(x)
   >         y = self.l2(y)
   >         ### other statements
   >         return y
   > ```
   > 
   > An example of the result of get_params.([MyModel](https://github.com/apache/singa/blob/5f485c0d21b107a119792f4a676009dcfad0d50f/proof_of_concept_to_remove.py))
   > 
   > ```shell
   > {'MyModel.l1.W':     [[-0.38202214 -0.12201381]
   >                       [-0.304533   -0.9495602 ]
   >                       [ 0.16352855 -0.01871952]
   >                       [ 0.8286818  -0.35981584]], 
   > 'MyModel.l1.b':       [0. 0.], 
   > 'MyModel.bn1.scale':  [1. 1.], 
   > 'MyModel.bn1.bias':   [0. 0.], 
   > 'MyModel.dl1.l1.W':  [[ 0.11860882 -0.79534566 -0.22841209  0.17164443]
   >                       [-0.6173898   0.66481847 -0.2978943   0.30236784]], 
   > 'MyModel.dl1.l1.b':   [0. 0. 0. 0.], 
   > 'MyModel.dl1.l2.W':  [[ 0.4442746  -1.1191332 ]
   >                       [ 0.38644505  0.8065585 ]
   >                       [-0.06753294 -0.19837534]
   >                       [-0.20991991  0.18191515]], 
   > 'MyModel.dl1.l2.b':   [0. 0.]}
   > ```
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message