The label tensor should be of shape (2947,) like this one
https://github.com/apache/incubatorsinga/blob/master/python/singa/metric.py#L31
I will check the problem from int32 and float32.
On Sun, Oct 9, 2016 at 3:27 PM Arash Shafiei <arash.shafiei@gmail.com>
wrote:
> I am facing a problem concerning creating tensor from numpy:
>
> Suppose that I have an array of these values:
> b = np.asarray([ [1., 0., 0., 0., 0., 0.], [0., 1., 0., 0., 0., 0.]])
>
> I transform it to a tensor:
> t = tensor.from_numpy(b.astype(np.int32))
>
> Now I again take the numpy back:
> a = tensor.to_numpy(t)
>
> The values of 'a' change and I lose accuracy:
> >>> a
> array([[ 1.40129846e45, 0.00000000e+00, 0.00000000e+00,
> 0.00000000e+00, 0.00000000e+00, 0.00000000e+00],
> [ 0.00000000e+00, 1.40129846e45, 0.00000000e+00,
> 0.00000000e+00, 0.00000000e+00, 0.00000000e+00]],
> dtype=float32)
>
> Now suppose that I create a tensor of float32:
> t = tensor.from_numpy(b.astype(np.float32))
>
> Now I again take the numpy back:
> a = tensor.to_numpy(t)
>
> Now the values of 'a' are correct:
> array([[ 1., 0., 0., 0., 0., 0.],
> [ 0., 1., 0., 0., 0., 0.]], dtype=float32)
>
> But in the second case where I am having float32, while calculating the L2
> norm:
> lvalue = lossfun.forward(model_pb2.kTrain, act, labels)
> batch_loss += lvalue.l2()
>
> The error message is:
> [F d1009 t15:26:37 p29915:584
> /home/wuwf/work/incubatorsinga/src/core/device/cuda_gpu.cc:112] Check
> failed: error == cudaSuccess (77 vs. 0) an illegal memory access was
> encountered
> Aborted (core dumped)
>
> Thanks for the support.
>
>
> On Sun, Oct 9, 2016 at 2:09 PM, Arash Shafiei <arash.shafiei@gmail.com>
> wrote:
>
> Thanks much.
>
> I am trying to use Evaluate() function. I pass activations of the dense
> layers and all the labels:
>
> evaluator.Evaluate(act,labels)
>
> The dimensions seem to be correct:
>
> act.shape == labels.shape == (2947, 6)
>
> But I am getting the following error:
> TypeError: in method 'Metric_Evaluate', argument 2 of type 'singa::Tensor
> const &'
>
> On Sun, Oct 9, 2016 at 1:17 PM, Wang Wei <wangwei.cs@gmail.com> wrote:
>
> No. Loss != Inaccuracy.
> If you want to compute the accuracy, you need to create an
> evaluator=singa.Accuracy(), and call evaluator.Evaluate(o, t), where o is
> the output from the dense layer and t is the ground truth tensor. You can
> follow the example here
> https://github.com/apache/incubatorsinga/blob/master/python/singa/metric.py#L67
> .
>
> Good Luck!
>
> On Sun, Oct 9, 2016 at 12:08 PM Arash Shafiei <arash.shafiei@gmail.com>
> wrote:
>
> Thanks for the hint.
>
> I was sending it to the device but the problem turned out to be that I did
> not cast labels to int32.
>
> Now it is working and I am getting:
>
> [................... ] 96.4% training loss = 0.003444
> Epoch 49, train loss is 0.003509
> Epoch 49, evaluation loss is 0.003534
>
> Does this mean that after 50 epoch the evaluation has only 3.5% inaccuracy?
>
> On Sun, Oct 9, 2016 at 11:29 AM, Wei Wang <wangwei.cs@gmail.com> wrote:
>
> Have you moved all tensor onto the same devices? Including the tensor for
> the labels.
>
>
> On 9 Oct 2016, at 11:02 AM, Arash Shafiei <arash.shafiei@gmail.com> wrote:
>
> outputs = rnn.forward(model_pb2.kTrain, inputs)[0:2]
> grads = []
> batch_loss = 0
> g_dense_w.set_value(0.0)
> g_dense_b.set_value(0.0)
> print 'outputs len', len(outputs) // 128
> output = outputs[1]
> act = dense.forward(model_pb2.kTrain, output)
> print 'output shape', output.shape // (256, 28)
> print 'activation shape', act.shape // (256, 6)
> print 'labels shape', labels.shape // (256, 6)
> lvalue = lossfun.forward(model_pb2.kTrain, act, labels)
> batch_loss += lvalue.l1() // [F d1009 t11:00:24 p23551:016
> /home/wuwf/work/incubatorsinga/src/core/tensor/./tensor_math_cuda.h:344]
> Check failed: status == CUBLAS_STATUS_SUCCESS (11 vs. 0)
> CUBLAS_STATUS_MAPPING_ERROR
> Aborted (core dumped)
>
>
>
>
> On Sun, Oct 9, 2016 at 10:55 AM, Wei Wang <wangwei.cs@gmail.com> wrote:
>
> Could you please paste the relevant code leading to this error?
>
>
>
> On 9 Oct 2016, at 10:32 AM, Arash Shafiei <arash.shafiei@gmail.com> wrote:
>
> Thanks, it worked.
>
> So far, I managed to do rnn::forward(...) but now I am stuck somewhere
> else.
>
> rnn::forward(...) returns a tensor (denoted as lvalue). I have to obtain
> the L1 norm using lvalue.l1().
>
> But I get this error:
> [F d1009 t10:30:14 p23056:56
> /home/wuwf/work/incubatorsinga/src/core/tensor/./tensor_math_cuda.h:344]
> Check failed: status == CUBLAS_STATUS_SUCCESS (11 vs. 0)
> CUBLAS_STATUS_MAPPING_ERROR
> Aborted (core dumped)
>
> On Sat, Oct 8, 2016 at 9:43 PM, Wang Wei <wangwei@comp.nus.edu.sg> wrote:
>
> Actually, the charrnn example is from type (4), where each rnn unit would
> generate a prediction and has a ground truth label.
>
> For your model (type 2), you only need to use the y128 (of shape 256, 28)
> from the rnn::forward() as the input to the dense layer. All other yi
> should be ignored.
> Consequently, you would have an output (denoted as o) of shape (256, 6)
> from the dense layer, which is the prediction for the whole sequence (of
> length 128).
> By feeding the prediction o and the label into the loss layer, you can
> compute the loss value and compute the gradient for o (denoted as o').
> Backward propagating the o through the dense layer, you would get the
> gradient for y128, denoted as y'128.
>
> *The input of the rnn::backward() would be <y'1, y'2, ...y'128, hy', cy'>,
> where only y'128 is a valid tensor. y'1, y'2 ... should be tensor with
> value 0.*
>
> Best,
> Wei
>
>
> On Sat, Oct 8, 2016 at 9:33 PM Arash Shafiei <arash.shafiei@gmail.com>
> wrote:
>
> Thanks. It worked.
>
> I am now at the phase of evaluating the loss.
>
> singa.loss.SoftmaxCrossEntropy has a forward function where it takes
> prediction tensors and ground truth.
>
> My problem now is that the prediction is a sequence and my label is not a
> sequence.
>
> Your charrnn example is an application of type (1) in the figure bellow,
> but activity recognition is an application of type (2).
>
>
> <rnnapp.png>
> Therefore for each sequence in a batch I have only 1 label. (although this
> label can be of one dimension from the set of {1,2,3,4,5,6} or of 6
> dimension from the set of { [1,0,0,0,0,0], [0,1,0,0,0,0] , etc. }
>
> So now I need predictions and ground truth. The prediction for me is of
> shape
> (128, 256, 28)
> where 128 is the length of the sequence, 256 is the batch size and 28 is
> the hidden layer size.
>
> And my ground truth is of shape
> (256, 1) or (256, 6)  depending on how you model it..
>
> But as I understood from the example of charrnn my ground truth must be
> of shape:
> (128, 256)
>
> Would you have any insight about this?
> Thanks..
>
>
> On Sat, Oct 8, 2016 at 6:42 PM, Wang Wei <wangwei@comp.nus.edu.sg> wrote:
>
> Currently, numpy array of dtype=np.float32 or np.int could be converted
> into singa tensor.
> Please convert the numpy array into np.float32 and then call
> tensor.from_numpy(t) (without dtype=np.float32).
>
> On Sat, Oct 8, 2016 at 6:36 PM Arash Shafiei <arash.shafiei@gmail.com>
> wrote:
>
> The values that I have are floating points [1 1].
>
> While using tensor.from_numpy(...), I was getting this error:
>
> Not implemented yet for float64
>
> I understood from the tutorial that we could pass the data type:
>
> y = tensor.from_numpy(..., dtype=np.float32)
>
> But using dtype, I am getting another error:
>
> TypeError: from_numpy() got an unexpected keyword argument 'dtype'
>
>
>
> On Sat, Oct 8, 2016 at 3:45 PM, Wang Wei <wangwei@comp.nus.edu.sg> wrote:
>
> Hi
>
> According to the API of forward function:
> http://singa.apache.org/en/docs/layer.html#singa.layer.RNN.forward
> The input should be a vector of Tensors, <x1, x2, ... x128, hx, cx>, xi is
> of shape (1500, 9), hx and cx are optional whose shape should be (1500, 28).
> The output would be a vector of Tensors, <y1, y2, ..., y128, hy, cy>, yi
> is of shape (1500, 28), hy and cy are optional depending on the existence
> of hx and cx.
> If you want to put the dense layer on top of the last rnn unit (i.e. the
> 128th), then you feed y128 to the dense layer.
>
> function convert just reshapes the raw data into a sequence of tensors
> <x1, x2, ..>.
>
> BTW, typically, people would use a smaller batchsize e.g. less than 256.
>
> May I forward our discussion to the incubator email list in case others
> have similar problems?
> Thanks.
>
> Best,
> Wei
>
> So here what I have:
>
> input batch of dimension (1500, 128, 9)
> This means a batch of 1500 windows each having 128 vector of 9 dimensions.
>
> input label of dimension (1500, 6)
> This means a label batch of 1500 vector of 6 dimensions. This is to label
> if the person is sitting ([1,0,0,0,0,0]) or standing ([0,1,0,0,0,0]), etc.
>
> I am creating an lstm layer with hidden_size=28 and
> input_sample_shape=(9,) and num_stacks=1
>
> Then I create a dense layer with num_output=6 and input_sample_shape=(28,)
>
> Now I would like to feed the data to the 'forward' function of lstm and
> dense layer. But I could not make it work and I could not quit understand
> from the example what 'convert' and 'numpy2tensors' are suppose to do...
>
> I would appreciate your comments..
>
> On Sun, Sep 25, 2016 at 12:23 PM, Arash Shafiei <arash.shafiei@gmail.com>
> wrote:
>
> Yes, I was thinking of batch size to be 32.
>
> Thanks. I am getting more how it works and I am thinking how RNN would be
> helpful. Because we do not want to predict a sequence. We just have a
> sequence (in raw data) and a set of features (in processed data) and we
> want to know the classification.
>
> So I was thinking of using other approaches with SINGA. I understood that
> there is also MLP. We could use MLP from SINGA to see the result first.
>
> In this case input would be a set of 561 values with a label.
> Then the MLP, given a set of test data with 561 features would predict the
> label.
>
> Thanks for advices..
>
>
>
