singa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <>
Subject [GitHub] [incubator-singa] chrishkchris edited a comment on issue #468: Distributted module
Date Thu, 15 Aug 2019 00:22:04 GMT
chrishkchris edited a comment on issue #468: Distributted module
   > Just now I did a test: When I replace the merged file by the latest master
branch, the loss can be reduced again. So I just changed the of dist_new
to the latest master branch one. This act would omit the commit f54a526 done on
   > I guess this maybe because:
   > The commit f54a526 in dist_new has redesigned the softmax function by calling cudnn,
but this is not compatiable with the latest master branch.
   OK, I check again in the AWS image, in the previous two months the code we used for distributed
module has commented out the code done by the commit f54a526 on softmax function, so softmax
is the same as master branch. This is the code of softmax we used in the AWS image of distributed
   void SoftMax(const Tensor &in, Tensor *out) {
     CHECK_LE(in.nDim(), 2u);
     size_t nrow = 1, ncol = in.Size(), size = ncol;
     if (in.nDim() == 2u) {
       nrow = in.shape(0);
       ncol = size / nrow;
       out->Reshape(Shape{nrow, ncol});
     Tensor tmp = RowMax(*out);
     SubColumn(tmp, out);
     Exp(*out, out);
     SumColumns(*out, &tmp);
     DivColumn(tmp, out);
   // void SoftMax(const Tensor &in, Tensor *out) {
   //   CHECK_LE(in.nDim(), 2u);
   //   TYPE_LANG_SWITCH(in.data_type(), DType, in.device()->lang(), Lang, {
   //     out->device()->Exec([in, out](Context * ctx) {
   //       SoftMax<DType, Lang>(in, out, ctx);
   //     }, {in.block(), out->block()}, {out->block()});
   //   });
   // }

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

With regards,
Apache Git Services

View raw message