singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-singa] chrishkchris opened a new pull request #543: SINGA-491 Use const reference when we use CopyData and ResetLike from Tensor Input
Date Thu, 03 Oct 2019 10:16:57 GMT
chrishkchris opened a new pull request #543: SINGA-491 Use const reference when we use CopyData
and ResetLike from Tensor Input
URL: https://github.com/apache/incubator-singa/pull/543
 
 
   This is to fix the remaining C language alert from LGTM: 
   
   There are some functions using ResetLike and CopyData to copy the data from the tensor
input. This PR changes the input type from "Tensor" to "const Tensor&"
   
   However, it seems that the src/model/layer is used by the old optimizer/model, it does
not have effect on the autograd.py based model.
   
   If I am correct, this avoid the system copying the input tensor with the tensor copy constructor,
instead it uses the address of the input pointer as reference.
   
   For a verification, the batchnorm in src/model/layer is used by the old optimizer (not
the autograd one), and hence we use the old resnet example to verify the training loss reduction
progress.
   
   ```
   ubuntu@ip-172-31-42-250:~/incubator-singa/examples/cifar10$ python3 train.py resnet cifar-10-batches-py
   Loading data ..................
   Loading data file cifar-10-batches-py/data_batch_1
   Loading data file cifar-10-batches-py/data_batch_2
   Loading data file cifar-10-batches-py/data_batch_3
   Loading data file cifar-10-batches-py/data_batch_4
   Loading data file cifar-10-batches-py/data_batch_5
   Loading data file cifar-10-batches-py/test_batch
   ('conv1', (16, 32, 32))
   ('bn1', (16, 32, 32))
   ('relu1', (16, 32, 32))
   ('2a-split', [(16, 32, 32), (16, 32, 32)])
   ('2a-br1-conv1', (16, 32, 32))
   ('2a-br1-bn1', (16, 32, 32))
   ('2a-br1-relu', (16, 32, 32))
   ('2a-br1-conv2', (16, 32, 32))
   ('2a-br1-bn2', (16, 32, 32))
   ('2a-merge', [(16, 32, 32), (16, 32, 32)])
   ('2b-split', [(16, 32, 32), (16, 32, 32)])
   ('2b-br1-conv1', (16, 32, 32))
   ('2b-br1-bn1', (16, 32, 32))
   ('2b-br1-relu', (16, 32, 32))
   ('2b-br1-conv2', (16, 32, 32))
   ('2b-br1-bn2', (16, 32, 32))
   ('2b-merge', [(16, 32, 32), (16, 32, 32)])
   ('2c-split', [(16, 32, 32), (16, 32, 32)])
   ('2c-br1-conv1', (16, 32, 32))
   ('2c-br1-bn1', (16, 32, 32))
   ('2c-br1-relu', (16, 32, 32))
   ('2c-br1-conv2', (16, 32, 32))
   ('2c-br1-bn2', (16, 32, 32))
   ('2c-merge', [(16, 32, 32), (16, 32, 32)])
   ('3a-split', [(16, 32, 32), (16, 32, 32)])
   ('3a-br2-conv', (32, 16, 16))
   ('3a-br2-bn', (32, 16, 16))
   ('3a-br1-conv1', (32, 16, 16))
   ('3a-br1-bn1', (32, 16, 16))
   ('3a-br1-relu', (32, 16, 16))
   ('3a-br1-conv2', (32, 16, 16))
   ('3a-br1-bn2', (32, 16, 16))
   ('3a-merge', [(32, 16, 16), (32, 16, 16)])
   ('3b-split', [(32, 16, 16), (32, 16, 16)])
   ('3b-br1-conv1', (32, 16, 16))
   ('3b-br1-bn1', (32, 16, 16))
   ('3b-br1-relu', (32, 16, 16))
   ('3b-br1-conv2', (32, 16, 16))
   ('3b-br1-bn2', (32, 16, 16))
   ('3b-merge', [(32, 16, 16), (32, 16, 16)])
   ('3c-split', [(32, 16, 16), (32, 16, 16)])
   ('3c-br1-conv1', (32, 16, 16))
   ('3c-br1-bn1', (32, 16, 16))
   ('3c-br1-relu', (32, 16, 16))
   ('3c-br1-conv2', (32, 16, 16))
   ('3c-br1-bn2', (32, 16, 16))
   ('3c-merge', [(32, 16, 16), (32, 16, 16)])
   ('4a-split', [(32, 16, 16), (32, 16, 16)])
   ('4a-br2-conv', (64, 8, 8))
   ('4a-br2-bn', (64, 8, 8))
   ('4a-br1-conv1', (64, 8, 8))
   ('4a-br1-bn1', (64, 8, 8))
   ('4a-br1-relu', (64, 8, 8))
   ('4a-br1-conv2', (64, 8, 8))
   ('4a-br1-bn2', (64, 8, 8))
   ('4a-merge', [(64, 8, 8), (64, 8, 8)])
   ('4b-split', [(64, 8, 8), (64, 8, 8)])
   ('4b-br1-conv1', (64, 8, 8))
   ('4b-br1-bn1', (64, 8, 8))
   ('4b-br1-relu', (64, 8, 8))
   ('4b-br1-conv2', (64, 8, 8))
   ('4b-br1-bn2', (64, 8, 8))
   ('4b-merge', [(64, 8, 8), (64, 8, 8)])
   ('4c-split', [(64, 8, 8), (64, 8, 8)])
   ('4c-br1-conv1', (64, 8, 8))
   ('4c-br1-bn1', (64, 8, 8))
   ('4c-br1-relu', (64, 8, 8))
   ('4c-br1-conv2', (64, 8, 8))
   ('4c-br1-bn2', (64, 8, 8))
   ('4c-merge', [(64, 8, 8), (64, 8, 8)])
   ('pool4', (64, 1, 1))
   ('flat', (64,))
   ('ip5', (10,))
   Start intialization............
   Start intialization............
   Using GPU
   Epoch=0: 100%|██████████████████████████████████████████████████████████████████████|
500/500 [00:17<00:00, 28.18it/s, accuracy=0.59, loss=1.13]
   Training loss = 1.418575, training accuracy = 0.481940, lr = 0.100000
   Test loss = 1.145096, test accuracy = 0.586800
   Epoch=1: 100%|█████████████████████████████████████████████████████████████████████|
500/500 [00:17<00:00, 29.34it/s, accuracy=0.76, loss=0.784]
   Training loss = 0.996122, training accuracy = 0.645940, lr = 0.100000
   Test loss = 0.947394, test accuracy = 0.665900
   Epoch=2: 100%|█████████████████████████████████████████████████████████████████████|
500/500 [00:17<00:00, 28.81it/s, accuracy=0.81, loss=0.696]
   Training loss = 0.812576, training accuracy = 0.713660, lr = 0.100000
   Test loss = 0.830808, test accuracy = 0.713700
   Epoch=3: 100%|█████████████████████████████████████████████████████████████████████|
500/500 [00:17<00:00, 27.96it/s, accuracy=0.81, loss=0.617]
   Training loss = 0.708455, training accuracy = 0.751980, lr = 0.100000
   Test loss = 0.761715, test accuracy = 0.740200
   Epoch=4: 100%|█████████████████████████████████████████████████████████████████████|
500/500 [00:17<00:00, 28.55it/s, accuracy=0.75, loss=0.737]
   Training loss = 0.636522, training accuracy = 0.777100, lr = 0.100000
   Test loss = 0.656281, test accuracy = 0.771600
   Epoch=5: 100%|█████████████████████████████████████████████████████████████████████|
500/500 [00:18<00:00, 27.43it/s, accuracy=0.78, loss=0.628]
   Training loss = 0.576065, training accuracy = 0.798540, lr = 0.100000
   Test loss = 0.642445, test accuracy = 0.783800
   Epoch=6: 100%|█████████████████████████████████████████████████████████████████████|
500/500 [00:17<00:00, 27.83it/s, accuracy=0.76, loss=0.584]
   Training loss = 0.530112, training accuracy = 0.814760, lr = 0.100000
   Test loss = 0.646009, test accuracy = 0.779800
   Epoch=7: 100%|█████████████████████████████████████████████████████████████████████|
500/500 [00:18<00:00, 28.51it/s, accuracy=0.83, loss=0.557]
   Training loss = 0.497382, training accuracy = 0.826420, lr = 0.100000
   Test loss = 0.602355, test accuracy = 0.791800
   Epoch=8: 100%|█████████████████████████████████████████████████████████████████████|
500/500 [00:17<00:00, 27.81it/s, accuracy=0.85, loss=0.427]
   Training loss = 0.457444, training accuracy = 0.840000, lr = 0.100000
   Test loss = 0.566916, test accuracy = 0.803500
   Epoch=9: 100%|█████████████████████████████████████████████████████████████████████|
500/500 [00:18<00:00, 27.70it/s, accuracy=0.83, loss=0.561]
   Training loss = 0.432371, training accuracy = 0.848380, lr = 0.100000
   Test loss = 0.616507, test accuracy = 0.798000
   Epoch=10: 100%|████████████████████████████████████████████████████████████████████|
500/500 [00:18<00:00, 28.38it/s, accuracy=0.89, loss=0.316]
   Training loss = 0.406027, training accuracy = 0.857000, lr = 0.100000
   Test loss = 0.615193, test accuracy = 0.798700
   Epoch=11: 100%|████████████████████████████████████████████████████████████████████|
500/500 [00:17<00:00, 27.92it/s, accuracy=0.88, loss=0.322]
   Training loss = 0.393299, training accuracy = 0.861660, lr = 0.100000
   Test loss = 0.572890, test accuracy = 0.811100
   Epoch=12: 100%|████████████████████████████████████████████████████████████████████|
500/500 [00:17<00:00, 28.69it/s, accuracy=0.88, loss=0.383]
   Training loss = 0.365866, training accuracy = 0.870500, lr = 0.100000
   Test loss = 0.602315, test accuracy = 0.803900
   Epoch=13: 100%|████████████████████████████████████████████████████████████████████|
500/500 [00:17<00:00, 27.98it/s, accuracy=0.86, loss=0.442]
   Training loss = 0.352359, training accuracy = 0.874200, lr = 0.100000
   Test loss = 0.666458, test accuracy = 0.790300
   Epoch=14: 100%|█████████████████████████████████████████████████████████████████████|
500/500 [00:17<00:00, 27.80it/s, accuracy=0.88, loss=0.33]
   Training loss = 0.328302, training accuracy = 0.882260, lr = 0.100000
   Test loss = 0.636559, test accuracy = 0.801700
   Epoch=15: 100%|████████████████████████████████████████████████████████████████████|
500/500 [00:17<00:00, 27.88it/s, accuracy=0.95, loss=0.238]
   Training loss = 0.313136, training accuracy = 0.889920, lr = 0.100000
   Test loss = 0.571674, test accuracy = 0.815300
   Epoch=16: 100%|████████████████████████████████████████████████████████████████████|
500/500 [00:18<00:00, 27.90it/s, accuracy=0.86, loss=0.364]
   Training loss = 0.298799, training accuracy = 0.894040, lr = 0.100000
   Test loss = 0.583324, test accuracy = 0.813600
   Epoch=17: 100%|█████████████████████████████████████████████████████████████████████|
500/500 [00:18<00:00, 27.65it/s, accuracy=0.91, loss=0.37]
   Training loss = 0.282553, training accuracy = 0.899700, lr = 0.100000
   Test loss = 0.641781, test accuracy = 0.805500
   Epoch=18: 100%|████████████████████████████████████████████████████████████████████|
500/500 [00:18<00:00, 26.91it/s, accuracy=0.91, loss=0.246]
   Training loss = 0.271136, training accuracy = 0.903680, lr = 0.100000
   Test loss = 0.619996, test accuracy = 0.809200
   Epoch=19: 100%|████████████████████████████████████████████████████████████████████|
500/500 [00:18<00:00, 26.87it/s, accuracy=0.93, loss=0.231]
   Training loss = 0.259176, training accuracy = 0.908360, lr = 0.100000
   Test loss = 0.579376, test accuracy = 0.822300
   Epoch=20: 100%|████████████████████████████████████████████████████████████████████|
500/500 [00:18<00:00, 27.60it/s, accuracy=0.92, loss=0.289]
   Training loss = 0.253127, training accuracy = 0.910180, lr = 0.100000
   Test loss = 0.649447, test accuracy = 0.806900
   ```
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message