singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [singa] dcslin commented on pull request #779: half float update
Date Wed, 26 Aug 2020 00:57:38 GMT

dcslin commented on pull request #779:
URL: https://github.com/apache/singa/pull/779#issuecomment-680390565


   tested below examples as a checkpoint:
   native.py with fp16
   ```
   root@1c6aaef3db53:~/singa-hp2# PYTHONPATH=build/python/ python3 examples/mlp/native.py
-pfloat16
   train_data_shape: (400, 2)
   train_label_shape: (400, 2)
   training loss =  0.6914
   training loss =  0.585
   training loss =  0.5596
   training loss =  0.539
   training loss =  0.4944
   training loss =  0.4238
   training loss =  0.319
   training loss =  0.2502
   training loss =  0.2102
   training loss =  0.1869
   training loss =  0.1671
   ```
   native.py with fp32
   ```
   root@1c6aaef3db53:~/singa-hp2# PYTHONPATH=build/python/ python3 examples/mlp/native.py

   train_data_shape: (400, 2)
   train_label_shape: (400, 2)
   training loss =  0.6908379
   training loss =  0.5781224
   training loss =  0.5531873
   training loss =  0.5157491
   training loss =  0.45046344
   training loss =  0.3674125
   training loss =  0.2854403
   training loss =  0.23216258
   training loss =  0.19450127
   training loss =  0.16646467
   training loss =  0.13695152
   ```
   
   
   module.py on fp16 with graph on
   ```
   root@1c6aaef3db53:~/singa-hp2# PYTHONPATH=build/python/ python3 examples/mlp/module.py
-pfloat16 
   WARNING: Logging before InitGoogleLogging() is written to STDERR
   F0826 00:48:40.063864 34058 tensor.cc:223] Check failed: block() && block()->initialized()
== true the data of the tensor needs be initialized before casting to another type
   *** Check failure stack trace: ***
   Aborted (core dumped)
   ```
   module.py on fp16 with graph off
   ```
   root@1c6aaef3db53:~/singa-hp2# PYTHONPATH=build/python/ python3 examples/mlp/module.py
-pfloat16 -g
   training loss =  0.6094
   training loss =  0.5225
   training loss =  0.467
   training loss =  0.404
   training loss =  0.3582
   training loss =  0.328
   training loss =  0.3164
   training loss =  0.3086
   training loss =  0.3108
   training loss =  0.3142
   training loss =  0.3198
   ```
   module.py on fp32 with graph on
   ```
   root@1c6aaef3db53:~/singa-hp2# PYTHONPATH=build/python/ python3 examples/mlp/module.py

   training loss =  0.61159235
   training loss =  0.5169311
   training loss =  0.43573818
   training loss =  0.34147996
   training loss =  0.26603624
   training loss =  0.21422084
   training loss =  0.17843087
   training loss =  0.15283388
   training loss =  0.13402645
   training loss =  0.11964666
   training loss =  0.10839656
   ```
   
   
   train cnn with mlp on fp16 with graph on
   ```
   root@1c6aaef3db53:~/singa-hp2# PYTHONPATH=build/python/ python3 examples/cnn/train_cnn.py
mlp mnist -m2 -pfloat16 
   Starting Epoch 0:
   WARNING: Logging before InitGoogleLogging() is written to STDERR
   F0826 00:49:13.757282 34338 tensor.cc:223] Check failed: block() && block()->initialized()
== true the data of the tensor needs be initialized before casting to another type
   *** Check failure stack trace: ***
   Aborted (core dumped)
   ```
   train cnn with mlp on fp16 with graph off
   ```
   root@1c6aaef3db53:~/singa-hp2# PYTHONPATH=build/python/ python3 examples/cnn/train_cnn.py
mlp mnist -m2 -pfloat16 -g
   Starting Epoch 0:
   Training loss = 449.630493, training accuracy = 0.869180
   Evaluation accuracy = 0.921675, Elapsed Time = 3.134102s
   Starting Epoch 1:
   Training loss = 250.288086, training accuracy = 0.925110
   Evaluation accuracy = 0.937200, Elapsed Time = 3.186108s
   root@1c6aaef3db53:~/singa-hp2# 
   ```
   train cnn with mlp on fp32 with graph off
   ```
   root@1c6aaef3db53:~/singa-hp2# PYTHONPATH=build/python/ python3 examples/cnn/train_cnn.py
mlp mnist -m2 -pfloat32 -g
   Starting Epoch 0:
   Training loss = 446.399231, training accuracy = 0.870331
   Evaluation accuracy = 0.922676, Elapsed Time = 2.745227s
   Starting Epoch 1:
   Training loss = 246.745819, training accuracy = 0.926194
   Evaluation accuracy = 0.938301, Elapsed Time = 2.591690s
   ```
   
   train cnn with cnn on fp16 with graph on
   ```
   root@1c6aaef3db53:~/singa-hp2# PYTHONPATH=build/python/ python3 examples/cnn/train_cnn.py
cnn mnist -m2 -pfloat16 
   Starting Epoch 0:
   WARNING: Logging before InitGoogleLogging() is written to STDERR
   F0826 00:49:58.988692 34502 tensor.cc:223] Check failed: block() && block()->initialized()
== true the data of the tensor needs be initialized before casting to another type
   *** Check failure stack trace: ***
   Aborted (core dumped)
   ```
   train cnn with cnn on fp16 with graph off
   ```
   root@1c6aaef3db53:~/singa-hp2# PYTHONPATH=build/python/ python3 examples/cnn/train_cnn.py
cnn mnist -m2 -pfloat16 -g
   Starting Epoch 0:
   Training loss = 599.249878, training accuracy = 0.788737
   Evaluation accuracy = 0.940104, Elapsed Time = 9.316158s
   Starting Epoch 1:
   Training loss = 236.738007, training accuracy = 0.920641
   Evaluation accuracy = 0.959335, Elapsed Time = 9.277672s
   ```
   train cnn with cnn on fp32 with graph off
   ```
   root@1c6aaef3db53:~/singa-hp2# PYTHONPATH=build/python/ python3 examples/cnn/train_cnn.py
cnn mnist -m2 -pfloat32 -g
   Starting Epoch 0:
   Training loss = 596.964600, training accuracy = 0.789421
   Evaluation accuracy = 0.943209, Elapsed Time = 8.189669s
   Starting Epoch 1:
   Training loss = 234.664322, training accuracy = 0.920758
   Evaluation accuracy = 0.960036, Elapsed Time = 8.101694s
   ```
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message