Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 6CC9C200B9F for ; Tue, 11 Oct 2016 14:56:57 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 6A8D9160AE6; Tue, 11 Oct 2016 12:56:57 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 608A1160AD2 for ; Tue, 11 Oct 2016 14:56:56 +0200 (CEST) Received: (qmail 82709 invoked by uid 500); 11 Oct 2016 12:56:55 -0000 Mailing-List: contact dev-help@singa.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@singa.incubator.apache.org Delivered-To: mailing list dev@singa.incubator.apache.org Received: (qmail 82698 invoked by uid 99); 11 Oct 2016 12:56:55 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Oct 2016 12:56:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 2C1631A0656 for ; Tue, 11 Oct 2016 12:56:55 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -7.019 X-Spam-Level: X-Spam-Status: No, score=-7.019 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id UQFW8vn1pRAM for ; Tue, 11 Oct 2016 12:56:52 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with SMTP id F14DA5F246 for ; Tue, 11 Oct 2016 12:56:51 +0000 (UTC) Received: (qmail 79956 invoked by uid 99); 11 Oct 2016 12:55:20 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 11 Oct 2016 12:55:20 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 91C6E2C4C70 for ; Tue, 11 Oct 2016 12:55:20 +0000 (UTC) Date: Tue, 11 Oct 2016 12:55:20 +0000 (UTC) From: "pangxiell (JIRA)" To: dev@singa.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (SINGA-260) Caffe layer featmap VS Singa layer featmap MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 11 Oct 2016 12:56:57 -0000 [ https://issues.apache.org/jira/browse/SINGA-260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15565335#comment-15565335 ] pangxiell commented on SINGA-260: --------------------------------- any interface directly change dense_conf.transpose() ? > Caffe layer featmap VS Singa layer featmap > ------------------------------------------- > > Key: SINGA-260 > URL: https://issues.apache.org/jira/browse/SINGA-260 > Project: Singa > Issue Type: Bug > Environment: ubuntu 14.04 singa1.0 cuda8.0 > Reporter: pangxiell > > HI Singa developer, please help me! Thank you! > I want to convert caffe trained model to singa. > After converting vgg16_deploy.prototxt and vgg16_trained.caffemodel to singa, > i test each layer featuremap between Caffe and Singa. I found something strage that same input , same weights same bias , but these two framework just get different result ! > All my computation done in CUDNN5.1 and cuda 8.0. > bellow is conv_1_1 featuremap and params and caffe net.prototxt > I am new to CNN, any help will be very pleasure. Thank you! > conv_1_1 output featuremap in Caffe : > [[[[ 0. 1.32925057 1.29131138 ..., 1.3344599 1.3344599 > 1.65958416] > [ 0. 0. 0. ..., 0. 0. 0. ] > [ 0. 0. 0. ..., 0. 0. 0. ] > ..., > [ 0. 0. 0. ..., 0. 0. 0. ] > [ 0. 0. 0. ..., 0. 0. 0. ] > [ 0. 0. 0. ..., 0. 0. 0. ]] > [[ 0. 0. 0. ..., 0. 0. 2.18781996] > [ 0. 1.65851605 1.64387 ..., 1.62676132 1.62676132 > 3.40298295] > [ 0. 1.71282542 1.80268919 ..., 1.62676132 1.62676132 > 3.40298295] > ..., > [ 0. 1.53555477 1.52216256 ..., 1.62676132 1.62676132 > 3.40298295] > [ 0. 1.51878488 1.54439628 ..., 1.62676132 1.62676132 > 3.40298295] > [ 0.45330271 2.44646335 2.46565628 ..., 2.5640552 2.5640552 > 3.23327875]] > [[ 0. 0. 0. ..., 0. 0. 0. ] > [ 0. 0. 0. ..., 0. 0. 0. ] > [ 0. 0. 0. ..., 0. 0. 0. ] > ..., > [ 0. 0. 0. ..., 0. 0. 0. ] > [ 0. 0. 0. ..., 0. 0. 0. ] > [ 0. 0. 0. ..., 0. 0. 0. ]] > ..., > [[ 0. 1.15822101 1.12836289 ..., 1.15823257 1.15823257 > 2.62147641] > [ 0. 0.32776707 0.3402763 ..., 0.32900044 0.32900044 > 4.58077908] > [ 0. 0.37061766 0.20697321 ..., 0.32900044 0.32900044 > 4.58077908] > ..., > [ 0. 0.29182461 0.26635617 ..., 0.32900044 0.32900044 > 4.58077908] > [ 0. 0.24839559 0.28039235 ..., 0.32900044 0.32900044 > 4.58077908] > [ 0. 0. 0. ..., 0. 0. 0.58074397]] > [[ 4.1525259 0.98238343 0.97233886 ..., 0.9321624 0.9321624 > 0.80502194] > [ 3.1123457 0. 0. ..., 0. 0. 0. ] > [ 3.10493231 0. 0. ..., 0. 0. 0. ] > ..., > [ 2.91378093 0. 0. ..., 0. 0. 0. ] > [ 2.90884662 0. 0. ..., 0. 0. 0. ] > [ 1.78133321 0. 0. ..., 0. 0. 0. ]] > [[ 4.23725843 3.92370176 3.87382889 ..., 3.92354441 3.92354441 > 1.62559974] > [ 5.06026077 5.14032888 5.14915466 ..., 5.25364971 5.25364971 > 3.15589523] > [ 4.98454475 5.13233662 5.07306051 ..., 5.25364971 5.25364971 > 3.15589523] > ..., > [ 4.92585659 5.04519129 5.08135509 ..., 5.25364971 5.25364971 > 3.15589523] > [ 4.95370245 5.03885031 5.08303356 ..., 5.25364971 5.25364971 > 3.15589523] > [ 3.74774575 4.78569221 4.83339024 ..., 5.0071063 5.0071063 > 1.9427563 ]]]] > conv1_1 weight of Caffe : > [[[[ 0.00850634 0.00754428 0.0104667 ] > [ 0.00282496 -0.0027129 0.00276441] > [-0.00195265 0.00433889 -0.00262078]] > [[ 0.0006124 -0.00466853 0.00038344] > [-0.00467937 0.00585917 0.00812106] > [ 0.00750268 -0.00080539 0.00109091]] > [[ 0.02275514 -0.00033002 -0.00400474] > [-0.00717515 0.01546651 -0.00067181] > [-0.01527147 -0.01406416 -0.0043859 ]]] > [[[ 0.01347855 -0.00352572 -0.00838911] > [ 0.00233354 0.005205 0.00387072] > [-0.00714711 0.00596602 0.01117658]] > [[-0.01879554 0.004475 -0.00248052] > [-0.00654012 -0.00028569 0.00818573] > [-0.01255068 0.01555492 -0.01060243]] > [[ 0.00812689 -0.01313079 -0.00390115] > [-0.01521709 -0.00033092 0.00772838] > [ 0.00827447 -0.01116868 0.00906964]]] > [[[-0.01065893 0.01276438 0.00548413] > [ 0.00571688 0.01419119 -0.00096069] > [ 0.00900401 0.00684223 0.00768703]] > [[-0.00515114 0.00051591 -0.00681952] > [ 0.00158043 0.01539363 0.00268879] > [-0.00661799 0.00388905 -0.01511758]] > [[ 0.00058433 0.00270303 -0.01194096] > [ 0.01680594 -0.00635558 -0.00678913] > [-0.00523596 0.00598517 -0.0058665 ]]] > ..., > [[[-0.00287419 -0.00421872 0.01433577] > [ 0.00379841 0.01539007 0.01394418] > [-0.00034302 -0.00515043 0.01400452]] > [[ 0.00740621 -0.00147624 0.00551567] > [-0.01737775 0.00381153 -0.00750302] > [-0.02425117 -0.00245259 -0.01052278]] > [[ 0.00449902 -0.01933564 0.0067264 ] > [ 0.00857356 -0.00080291 -0.00353165] > [-0.00313204 -0.00219617 0.00922761]]] > [[[-0.00592723 0.00357399 -0.00755919] > [-0.00429552 -0.00372033 0.00735837] > [ 0.00408205 -0.00690898 -0.00823291]] > [[ 0.00788845 0.02441542 -0.00630782] > [ 0.01927572 -0.00438485 -0.00568748] > [ 0.00435551 -0.006596 0.00158271]] > [[ 0.00721608 -0.00344833 0.00014568] > [ 0.00832353 -0.0067124 -0.010458 ] > [-0.00227656 -0.00796929 0.01343615]]] > [[[-0.00255021 0.02384895 -0.00994168] > [-0.00690358 -0.0033091 -0.00871819] > [-0.00353378 0.00959968 -0.00176385]] > [[-0.01070219 -0.01495273 -0.00126908] > [-0.00315238 0.00230987 -0.00847946] > [ 0.00717464 0.00052554 0.00878843]] > [[ 0.00727798 -0.0137497 0.0097814 ] > [ 0.00240273 0.0024867 -0.01207745] > [ 0.0045142 -0.02454388 0.00163318]]]] > conv1_1 bias of Caffe : > [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. > 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. > 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. > 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] > conv_1_1 output featuremap in Singa : > [[[[-1.15990531 1.32925057 1.29131138 ..., 1.3344599 1.3344599 > 1.65958416] > [-1.92470229 -3.24402356 -3.19016767 ..., -3.14230347 -3.14230347 > -2.3147471 ] > [-2.00432348 -3.56044745 -3.69492149 ..., -3.14230347 -3.14230347 > -2.3147471 ] > ..., > [-1.73503172 -2.94724107 -3.00977921 ..., -3.14230347 -3.14230347 > -2.3147471 ] > [-1.73839962 -2.96330857 -2.96887565 ..., -3.14230347 -3.14230347 > -2.3147471 ] > [-3.8452611 -6.39361048 -6.45467329 ..., -6.70470476 -6.70470476 > -5.17954683]] > [[-4.45994806 -1.13598835 -1.14740443 ..., -1.13741207 -1.13741207 > 2.18781996] > [-1.40371311 1.65851605 1.64387 ..., 1.62676132 1.62676132 > 3.40298295] > [-1.35754728 1.71282542 1.80268919 ..., 1.62676132 1.62676132 > 3.40298295] > ..., > [-1.26869226 1.53555477 1.52216256 ..., 1.62676132 1.62676132 > 3.40298295] > [-1.30324793 1.51878488 1.54439628 ..., 1.62676132 1.62676132 > 3.40298295] > [ 0.45330271 2.44646335 2.46565628 ..., 2.5640552 2.5640552 > 3.23327875]] > [[-1.58399725 -3.95110941 -3.98022842 ..., -3.97811508 -3.97811508 > -6.28557253] > [-1.47556436 -2.40914106 -2.4150188 ..., -2.37049937 -2.37049937 > -6.42171335] > [-1.64208531 -2.60111237 -2.59987879 ..., -2.37049937 -2.37049937 > -6.42171335] > ..., > [-1.31520605 -2.15799737 -2.18744612 ..., -2.37049937 -2.37049937 > -6.42171335] > [-1.32342124 -2.1652441 -2.22677469 ..., -2.37049937 -2.37049937 > -6.42171335] > [-1.14510441 -2.52267647 -2.55242896 ..., -2.69865441 -2.69865441 > -5.12770367]] > ..., > [[-2.15458727 1.15822101 1.12836289 ..., 1.15823257 1.15823257 > 2.62147641] > [-1.88057959 0.32776707 0.3402763 ..., 0.32900044 0.32900044 > 4.58077908] > [-1.8912673 0.37061766 0.20697321 ..., 0.32900044 0.32900044 > 4.58077908] > ..., > [-1.80236626 0.29182461 0.26635617 ..., 0.32900044 0.32900044 > 4.58077908] > [-1.77869821 0.24839559 0.28039235 ..., 0.32900044 0.32900044 > 4.58077908] > [-1.43230343 -2.1616652 -2.18854642 ..., -2.27060723 -2.27060723 > 0.58074397]] > [[ 4.1525259 0.98238343 0.97233886 ..., 0.9321624 0.9321624 > 0.80502194] > [ 3.1123457 -1.32611859 -1.39405644 ..., -1.36906409 -1.36906409 > -2.84570265] > [ 3.10493231 -1.32919359 -1.42399073 ..., -1.36906409 -1.36906409 > -2.84570265] > ..., > [ 2.91378093 -1.27367365 -1.24672067 ..., -1.36906409 -1.36906409 > -2.84570265] > [ 2.90884662 -1.2343148 -1.26084697 ..., -1.36906409 -1.36906409 > -2.84570265] > [ 1.78133321 -1.90749574 -1.9271915 ..., -2.06096601 -2.06096601 > -4.66862774]] > [[ 4.23725843 3.92370176 3.87382889 ..., 3.92354441 3.92354441 > 1.62559974] > [ 5.06026077 5.14032888 5.14915466 ..., 5.25364971 5.25364971 > 3.15589523] > [ 4.98454475 5.13233662 5.07306051 ..., 5.25364971 5.25364971 > 3.15589523] > ..., > [ 4.92585659 5.04519129 5.08135509 ..., 5.25364971 5.25364971 > 3.15589523] > [ 4.95370245 5.03885031 5.08303356 ..., 5.25364971 5.25364971 > 3.15589523] > [ 3.74774575 4.78569221 4.83339024 ..., 5.0071063 5.0071063 > 1.9427563 ]]]] > conv1_1_weight of singa [[ 0.00850634 0.00754428 0.0104667 ..., -0.01527147 -0.01406416 > -0.0043859 ] > [ 0.01347855 -0.00352572 -0.00838911 ..., 0.00827447 -0.01116868 > 0.00906964] > [-0.01065893 0.01276438 0.00548413 ..., -0.00523596 0.00598517 > -0.0058665 ] > ..., > [-0.00287419 -0.00421872 0.01433577 ..., -0.00313204 -0.00219617 > 0.00922761] > [-0.00592723 0.00357399 -0.00755919 ..., -0.00227656 -0.00796929 > 0.01343615] > [-0.00255021 0.02384895 -0.00994168 ..., 0.0045142 -0.02454388 > 0.00163318]] > conv1_1_bias of singa [ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. > 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. > 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] > caffe prototxt: > name: "VGG_ILSVRC_16_layers" > input: "data" > input_dim: 10 > input_dim: 3 > input_dim: 224 > input_dim: 224 > layers { > bottom: "data" > top: "conv1_1" > name: "conv1_1" > type: CONVOLUTION > convolution_param { > num_output: 64 > pad: 1 > kernel_size: 3 > } > } > layers { > bottom: "conv1_1" > top: "conv1_1" > name: "relu1_1" > type: RELU > } > layers { > bottom: "conv1_1" > top: "conv1_2" > name: "conv1_2" > type: CONVOLUTION > convolution_param { > num_output: 64 > pad: 1 > kernel_size: 3 > } > } > layers { > bottom: "conv1_2" > top: "conv1_2" > name: "relu1_2" > type: RELU > } > layers { > bottom: "conv1_2" > top: "pool1" > name: "pool1" > type: POOLING > pooling_param { > pool: MAX > kernel_size: 2 > stride: 2 > } > } > layers { > bottom: "pool1" > top: "conv2_1" > name: "conv2_1" > type: CONVOLUTION > convolution_param { > num_output: 128 > pad: 1 > kernel_size: 3 > } > } > layers { > bottom: "conv2_1" > top: "conv2_1" > name: "relu2_1" > type: RELU > } > layers { > bottom: "conv2_1" > top: "conv2_2" > name: "conv2_2" > type: CONVOLUTION > convolution_param { > num_output: 128 > pad: 1 > kernel_size: 3 > } > } > layers { > bottom: "conv2_2" > top: "conv2_2" > name: "relu2_2" > type: RELU > } > layers { > bottom: "conv2_2" > top: "pool2" > name: "pool2" > type: POOLING > pooling_param { > pool: MAX > kernel_size: 2 > stride: 2 > } > } > layers { > bottom: "pool2" > top: "conv3_1" > name: "conv3_1" > type: CONVOLUTION > convolution_param { > num_output: 256 > pad: 1 > kernel_size: 3 > } > } > layers { > bottom: "conv3_1" > top: "conv3_1" > name: "relu3_1" > type: RELU > } > layers { > bottom: "conv3_1" > top: "conv3_2" > name: "conv3_2" > type: CONVOLUTION > convolution_param { > num_output: 256 > pad: 1 > kernel_size: 3 > } > } > layers { > bottom: "conv3_2" > top: "conv3_2" > name: "relu3_2" > type: RELU > } > layers { > bottom: "conv3_2" > top: "conv3_3" > name: "conv3_3" > type: CONVOLUTION > convolution_param { > num_output: 256 > pad: 1 > kernel_size: 3 > } > } > layers { > bottom: "conv3_3" > top: "conv3_3" > name: "relu3_3" > type: RELU > } > layers { > bottom: "conv3_3" > top: "pool3" > name: "pool3" > type: POOLING > pooling_param { > pool: MAX > kernel_size: 2 > stride: 2 > } > } > layers { > bottom: "pool3" > top: "conv4_1" > name: "conv4_1" > type: CONVOLUTION > convolution_param { > num_output: 512 > pad: 1 > kernel_size: 3 > } > } > layers { > bottom: "conv4_1" > top: "conv4_1" > name: "relu4_1" > type: RELU > } > layers { > bottom: "conv4_1" > top: "conv4_2" > name: "conv4_2" > type: CONVOLUTION > convolution_param { > num_output: 512 > pad: 1 > kernel_size: 3 > } > } > layers { > bottom: "conv4_2" > top: "conv4_2" > name: "relu4_2" > type: RELU > } > layers { > bottom: "conv4_2" > top: "conv4_3" > name: "conv4_3" > type: CONVOLUTION > convolution_param { > num_output: 512 > pad: 1 > kernel_size: 3 > } > } > layers { > bottom: "conv4_3" > top: "conv4_3" > name: "relu4_3" > type: RELU > } > layers { > bottom: "conv4_3" > top: "pool4" > name: "pool4" > type: POOLING > pooling_param { > pool: MAX > kernel_size: 2 > stride: 2 > } > } > layers { > bottom: "pool4" > top: "conv5_1" > name: "conv5_1" > type: CONVOLUTION > convolution_param { > num_output: 512 > pad: 1 > kernel_size: 3 > } > } > layers { > bottom: "conv5_1" > top: "conv5_1" > name: "relu5_1" > type: RELU > } > layers { > bottom: "conv5_1" > top: "conv5_2" > name: "conv5_2" > type: CONVOLUTION > convolution_param { > num_output: 512 > pad: 1 > kernel_size: 3 > } > } > layers { > bottom: "conv5_2" > top: "conv5_2" > name: "relu5_2" > type: RELU > } > layers { > bottom: "conv5_2" > top: "conv5_3" > name: "conv5_3" > type: CONVOLUTION > convolution_param { > num_output: 512 > pad: 1 > kernel_size: 3 > } > } > layers { > bottom: "conv5_3" > top: "conv5_3" > name: "relu5_3" > type: RELU > } > layers { > bottom: "conv5_3" > top: "pool5" > name: "pool5" > type: POOLING > pooling_param { > pool: MAX > kernel_size: 2 > stride: 2 > } > } > layers { > name: "flatdata" > type: FLATTEN > bottom: "pool5" > top: "flatdata" > } > layers { > bottom: "flatdata" > top: "fc6" > name: "fc6" > type: INNER_PRODUCT > inner_product_param { > num_output: 4096 > } > } > layers { > bottom: "fc6" > top: "fc6" > name: "relu6" > type: RELU > } > layers { > bottom: "fc6" > top: "fc6" > name: "drop6" > type: DROPOUT > dropout_param { > dropout_ratio: 0.5 > } > } > layers { > bottom: "fc6" > top: "fc7" > name: "fc7" > type: INNER_PRODUCT > inner_product_param { > num_output: 4096 > } > } > layers { > bottom: "fc7" > top: "fc7" > name: "relu7" > type: RELU > } > layers { > bottom: "fc7" > top: "fc7" > name: "drop7" > type: DROPOUT > dropout_param { > dropout_ratio: 0.5 > } > } > layers { > bottom: "fc7" > top: "fc8" > name: "fc8" > type: INNER_PRODUCT > inner_product_param { > num_output: 1000 > } > } > layers { > bottom: "fc8" > top: "prob" > name: "prob" > type: SOFTMAX > } -- This message was sent by Atlassian JIRA (v6.3.4#6332)