singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <>
Subject [jira] [Commented] (SINGA-100) Implement layers using CUDNN for GPU training
Date Wed, 16 Dec 2015 12:11:46 GMT


ASF subversion and git services commented on SINGA-100:

Commit f3b47c706e09315f2eb8eb1d599597093c980962 in incubator-singa's branch refs/heads/master
from [~flytosky]
[;h=f3b47c7 ]

SINGA-100 Implement layers using CUDNN for GPU training

Replace CopyFrom with Copy in softmax loss layers. CopyFrom copies both
cpu and gpu memory, while Copy copies either cpu or gpu memory.

Rename CudnnLayer to CudnnBase, which was changed but missed during rebase..

Fix bugs from getting stuck using GPU (CUDA) operations.
Possible causes include,
1. different threads access the same GPU memory addr. The non-owner thread would get stuck.
2. cudnnCreate gets stuck if the GPU device is used by other programs.
3. CudnnSoftmaxLossLayer::Setup forgets to call LossLayer::Setup which leaves layer_conf_

1. move cudnn headers and libs before cuda in compiling, otherwise there would be
link conflicts if there are multiple versions of cudnn in the system.
2. replace malloc with cudaMalloc in which pins the CPU
memory for efficient and stable memory transferring between CPU and GPU.

Tested with multiple GPUs and single GPU.

> Implement layers using CUDNN for GPU training
> ---------------------------------------------
>                 Key: SINGA-100
>                 URL:
>             Project: Singa
>          Issue Type: New Feature
>            Reporter: wangwei
> NVIDIA has released the cudnn library optimized for CNN operations like convolution,
pooling, etc. It has achieved overall good performance. Hence, it is essential to add cudnn
supported layers in SINGA for efficient GPU training (SINGA-41).
> We will use the cudnn library to implement CNN layers, namely,
>  cudnnConvolutionLayer, cudnnPoolingLayer, cudnnLRNLayer, cudnnSoftmaxLayer, cudnnReLULayer,
cudnnSigmoidLayer, cudnnTanhLayer, cudnnDivNormLayer.
> Data type float-16 will not be consider in this ticket.

This message was sent by Atlassian JIRA

View raw message