mxnet-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] lawrencewxj opened a new issue #8832: I didn't get a reasonable speed-up when applying depthwise convolution to VGG16
Date Thu, 01 Jan 1970 00:00:00 GMT
lawrencewxj opened a new issue #8832: I didn't get a reasonable speed-up when applying depthwise
convolution to VGG16
URL: https://github.com/apache/incubator-mxnet/issues/8832
 
 
   ## Description
   It should speed up for about 8x in theory?but it seems get a litter bit lower in practice.
Is there something wrong in my code?   @crazy-cat 
   **the original VGG convolution:**
   ```
   internel_layer = mx.sym.Convolution(data = internel_layer, kernel=(3, 3), pad=(1, 1), num_filter=filters[i],
name="conv%s_%s" %(i + 1, j + 1))
   internel_layer = mx.symbol.BatchNorm(data=internel_layer, name="bn%s_%s" %(i + 1, j + 1))
   internel_layer = mx.sym.Activation(data=internel_layer, act_type="relu", name="relu%s_%s"
%(i + 1, j + 1))
   ```
   **after apply depthwise convolution**
   ```
   internel_layer = mx.sym.Convolution(data = internel_layer, kernel=(3, 3), pad=(1, 1),
   num_filter=input_featuremap_channel, num_group=input_featuremap_channel, name="a_conv%s_%s"
%(i + 1, j + 1))
   internel_layer = mx.sym.Convolution(data=internel_layer, kernel=(1, 1), num_filter=filters[i],
name="c_conv%s_%s" % (i + 1, j + 1))
   internel_layer = mx.symbol.BatchNorm(data=internel_layer, name="c_bn%s_%s" % (i + 1, j
+ 1))
   internel_layer = mx.sym.Activation(data=internel_layer, act_type="relu", name="c_relu%s_%s"
% (i + 1, j + 1))
   ```
   **the original VGG convolution speed:**
   ```
   INFO:root:start with arguments Namespace(batch_size=32, benchmark=0, data_nthreads=8, data_train='/home/code/wangxijun/data/webface/scene_train.rec',
data_val=None, disp_batches=20, dtype='float32', gpus='2', image_shape='3,224,224', kv_store='device',
load_epoch=None, lr=0.01, lr_factor=0.1, lr_step_epochs='20,50,90', max_random_aspect_ratio=0.25,
max_random_h=36, max_random_l=50, max_random_rotate_angle=10, max_random_s=50, max_random_scale=1,
max_random_shear_ratio=0.1, min_random_scale=1, model_prefix='/home/code/wangxijun/model/vgg1/vgg',
mom=0.9, monitor=0, network='vgg', num_classes=10575, num_epochs=400, num_examples=477362,
num_layers=16, optimizer='sgd', pad_size=0, random_crop=1, random_mirror=1, rgb_mean='123.68,116.779,103.939',
test_io=0, top_k=0, wd=0.0001)
   [17:56:13] src/io/iter_image_recordio_2.cc:153: ImageRecordIOParser2: /home/code/wangxijun/data/webface/scene_train.rec,
use 5 threads for decoding..
   [17:56:23] src/operator/././cudnn_algoreg-inl.h:112: Running performance tests to find
the best convolution algorithm, this can take a while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT
to 0 to disable)
   INFO:root:Epoch[0] Batch [20]   Speed: 33.19 samples/sec        cross-entropy=12.565712
   INFO:root:Epoch[0] Batch [40]   Speed: 33.20 samples/sec        cross-entropy=10.379888
   INFO:root:Epoch[0] Batch [60]   Speed: 32.95 samples/sec        cross-entropy=9.781008
   INFO:root:Epoch[0] Batch [80]   Speed: 32.94 samples/sec        cross-entropy=9.505107
   INFO:root:Epoch[0] Batch [100]  Speed: 32.50 samples/sec        cross-entropy=9.321202
   INFO:root:Epoch[0] Batch [120]  Speed: 32.33 samples/sec        cross-entropy=9.279530
   INFO:root:Epoch[0] Batch [140]  Speed: 32.52 samples/sec        cross-entropy=9.288194
   INFO:root:Epoch[0] Batch [160]  Speed: 32.27 samples/sec        cross-entropy=9.287867
   INFO:root:Epoch[0] Batch [180]  Speed: 32.19 samples/sec        cross-entropy=9.276982
   INFO:root:Epoch[0] Batch [200]  Speed: 31.90 samples/sec        cross-entropy=9.267534
   INFO:root:Epoch[0] Batch [220]  Speed: 32.26 samples/sec        cross-entropy=9.266759
   ```
   **after apply depthwise convolution speed:**
   ```
   INFO:root:start with arguments Namespace(batch_size=32, benchmark=0, data_nthreads=8, data_train='/home/code/wangxijun/data/webface/scene_train.rec',
data_val=None, disp_batches=20, dtype='float32', gpus='3', image_shape='3,224,224', kv_store='device',
load_epoch=None, lr=0.01, lr_factor=0.1, lr_step_epochs='20,50,90', max_random_aspect_ratio=0.25,
max_random_h=36, max_random_l=50, max_random_rotate_angle=10, max_random_s=50, max_random_scale=1,
max_random_shear_ratio=0.1, min_random_scale=1, model_prefix='/home/code/wangxijun/model/vgg2/vgg',
mom=0.9, monitor=0, network='vgg', num_classes=10575, num_epochs=400, num_examples=477362,
num_layers=16, optimizer='sgd', pad_size=0, random_crop=1, random_mirror=1, rgb_mean='123.68,116.779,103.939',
test_io=0, top_k=0, wd=0.0001)
   [17:57:20] src/io/iter_image_recordio_2.cc:153: ImageRecordIOParser2: /home/code/wangxijun/data/webface/scene_train.rec,
use 5 threads for decoding..
   [17:57:34] src/operator/././cudnn_algoreg-inl.h:112: Running performance tests to find
the best convolution algorithm, this can take a while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT
to 0 to disable)
   INFO:root:Epoch[0] Batch [20]   Speed: 31.45 samples/sec        cross-entropy=14.358711
   INFO:root:Epoch[0] Batch [40]   Speed: 32.21 samples/sec        cross-entropy=10.281333
   INFO:root:Epoch[0] Batch [60]   Speed: 32.16 samples/sec        cross-entropy=9.439612
   INFO:root:Epoch[0] Batch [80]   Speed: 32.09 samples/sec        cross-entropy=9.352679
   INFO:root:Epoch[0] Batch [100]  Speed: 31.95 samples/sec        cross-entropy=9.296035
   INFO:root:Epoch[0] Batch [120]  Speed: 31.97 samples/sec        cross-entropy=9.260700
   INFO:root:Epoch[0] Batch [140]  Speed: 32.08 samples/sec        cross-entropy=9.323341
   INFO:root:Epoch[0] Batch [160]  Speed: 31.94 samples/sec        cross-entropy=9.229615
   INFO:root:Epoch[0] Batch [180]  Speed: 32.03 samples/sec        cross-entropy=9.285774
   INFO:root:Epoch[0] Batch [200]  Speed: 31.97 samples/sec        cross-entropy=9.259684
   INFO:root:Epoch[0] Batch [220]  Speed: 31.92 samples/sec        cross-entropy=9.250173
   ```
   **Hardware :**
   K80
   **Software :**
   cuda8.0 + cudnn5.1
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message