singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [singa] dcslin commented on pull request #733: add imdb sentiment model
Date Mon, 15 Jun 2020 02:45:03 GMT

dcslin commented on pull request #733:
URL: https://github.com/apache/singa/pull/733#issuecomment-643872776


   > I took a look at your cudnn rnn function:
   > https://github.com/apache/singa/blob/dev/python/singa/layer.py#L1506
   > you transpose the input so that "inputs has shape of {sequence length, batch size,
feature size}"
   > 
   > Meanwhile, when I read the cudnn API `cudnnRNNForwardInference()`
   > https://docs.nvidia.com/deeplearning/sdk/cudnn-api/index.html#cudnnRNNForwardInference
   > the description of x is:
   > 
   > _xDesc Input. An array of seqLength fully packed tensor descriptors. Each descriptor
in the array should have three dimensions that describe the input data format to one recurrent
iteration (one descriptor per RNN time-step). **The first dimension (batch size)** of the
tensors may decrease from iteration n to iteration n+1 but may not increase. Each tensor descriptor
must have the same second dimension (RNN input vector length, inputSize). The third dimension
of each tensor should be 1. Input data are expected to be arranged in the column-major order
so strides in xDesc should be set as follows:_
   > 
   > See the highlighted text in the above description. The first dimension is batch size?
   > 
   > It is a bit confusing for me, so I am not sure what should be the input shape. I suggest
checking all those input output format if you don't have further idea for debug.
   
   thank you for the review, this part is indeed confusing.
   
   ### in the general case of cudnn api
   so we need to pass a list of xDesc into cudnn
   which means xDesc0, xDesc1, xDesc2, xDesc3, ... xDescn
   each xDesc has shape of {bs, input}
   means xDesc0{bs1, input}, xDesc2{bs2, input}, xDesc3{bs3, input} ... xDescn{bsn, input}
   and the total length of these xDesc0, xDesc1, xDesc2, xDesc3, xDescn is seq length
   
   ### in the special case assuming all the `bs` are the same
   for data pointer, singa tensor x, we will have shape {seq_length, bs, input}


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message