Mailing-List: contact dev-help@horn.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@horn.incubator.apache.org
From: "Edward J. Yoon" <edward.yoon@samsung.com>
To: dev@horn.incubator.apache.org
References: <000301d137d3$221dcae0$665960a0$@samsung.com>
 <CAJp0o0WEszq_Hkc2dTLgBkHMuFA6HLFTDOJXDx08az3r-_=6MA@mail.gmail.com>
 <000401d13872$d078a610$7169f230$@samsung.com>
 <CAJp0o0VyA=0SOwidN5cS2ztefNFMy4nQFh-JN+AapSXQMYfMgg@mail.gmail.com>
 <002f01d15278$a1f90620$e5eb1260$@samsung.com>
 <CAJp0o0X-AjA85xtSWWUGWa9yJPqhumqR2vqpYUX94yqOKn7uAw@mail.gmail.com>
 <002701d1531b$974fb220$c5ef1660$@samsung.com>
In-reply-to: <002701d1531b$974fb220$c5ef1660$@samsung.com>
Subject: RE: Implement convolutional neuron function
Date: Thu, 21 Apr 2016 13:58:17 +0900
Message-id: <001901d19b8a$6b153880$413fa980$@samsung.com>
Thread-index: 
 AQIhjqZh6m3ILEspgfHXXVuHNTXULQEGQg/VAsbJZ0QCv1koXgJXbTR9AmfvVRwCr0S+JJ6EH0ew
Content-language: ko
DLP-Filter: Pass

Ping, you there? Let's re-start :-)

--
Best Regards, Edward J. Yoon


-----Original Message-----
From: Edward J. Yoon [mailto:edward.yoon@samsung.com]
Sent: Wednesday, January 20, 2016 9:44 AM
To: dev@horn.incubator.apache.org
Subject: RE: Implement convolutional neuron function

Message type is flexible. You can create your own message type for example,
FeatureVectorMessage.

> where the summation part happens later?

As I mentioned, the similar pseudo code can be found at
https://cwiki.apache.org/confluence/display/HORN/Programming+APIs

Input message is the feature map as [x1, x2, ..., xN], the convolution
computation can be represented like:

Upward() {
  for each [x1, x2, ..., xN] do
    y += w * x

  propagate(ReLu(y));
}

Then, we train the filter weights at downward() method. Single neuron is
equal
to single element of matrix.

I recommend you to implement only upward and downward method of convolution
neuron at the moment.

--
Best Regards, Edward J. Yoon

-----Original Message-----
From: Zachary Jaffee [mailto:zij@case.edu]
Sent: Wednesday, January 20, 2016 5:18 AM
To: Unknown
Subject: Re: Implement convolutional neuron function

Ok, so I think I have a clearer idea now, but just to confirm some things,
the PropMessage in the code you have written represents the user defined
neuron function? Additionally, does the neuron object continue to store the
result as the element-wise multiplication matrix, i.e. as the hadamard
product result, where the summation part happens later?

On Mon, Jan 18, 2016 at 9:16 PM, Edward J. Yoon <edward.yoon@samsung.com>
wrote:

> Hi,
>
> As described in "Convolution Demo" section of
> http://cs231n.github.io/convolutional-networks, convolution layer
> construct
> output maps by convoluting trainable kernel filter. Animation will be very
> helpful. The initial weights of kernel filter are random like MLP. And,
> this
> convolution computation between a feature map and a kernel can be
> simplified
> to a vector/matrix multiplication. The batch of multiple images is mat-mat
> multiplication. This is normal way in other projects like Torch.
>
> Instead we doing like that, we'll do element-wise multiplication within
> each
> neuron object. User define the neuron function then, framework process the
> neurons in parallel. To understand this flow, Pregel system will be
> helpful.
>
> I roughly guess our system can be useful when if a server receive image to
> recognize. Because, GPU-oriented systems are optimized to process batch
> operations.
>
> --
> Best Regards, Edward J. Yoon
>
>
> -----Original Message-----
> From: Zachary Jaffee [mailto:zij@case.edu]
> Sent: Tuesday, January 19, 2016 1:34 PM
> To: Unknown
> Subject: Re: Implement convolutional neuron function
>
> So I've tried out a few things, but I can't seem to see what you mean when
> you say not to use a dot product. If you have any insights as to what this
> function would look like, I'd be interested in seeing what you are
thinking
> of.
>
> On Wed, Dec 16, 2015 at 6:29 PM, Edward J. Yoon <edward.yoon@samsung.com>
> wrote:
>
> > Thanks.
> >
> > You can discuss this issue with me and shubham.
> >
> > I personally we need to approach neuron-centric instead of dot product
of
> > matrix or tensor, so that we can parallelize computations at neuron
> level.
> > In
> > here, the tricky issue is handling the topology of neurons within
> > rectangular
> > grid (graph structure). But, you can ignore this at the moment.
> >
> > If you have any questions/opinions, let's discuss together.
> >
> > --
> > Best Regards, Edward J. Yoon
> >
> > -----Original Message-----
> > From: Zachary Jaffee [mailto:zij@case.edu]
> > Sent: Thursday, December 17, 2015 12:59 AM
> > To: Unknown
> > Subject: Re: Implement convolutional neuron function
> >
> > I'll take care of this.
> >
> > On Wed, Dec 16, 2015 at 2:26 AM, Edward J. Yoon <edward.yoon@samsung.com
> >
> > wrote:
> >
> > > Hi forks,
> > >
> > > Does anyone volunteer for HORN-10?
> > >
> > > To implement convolutional neuron function, you can refer my standard
> > > neuron
> > > function code here:
> > >
> > >
> >
>
https://github.com/edwardyoon/incubator-horn/blob/HORN-7/src/test/java/org/a
> > > pache/horn/trainer/TestNeuron.java
> > >
> > > And basic equation
> > >
> > >
> >
>
https://jianfengwang.files.wordpress.com/2015/07/forwardandbackwardpropagati
> > > onofconvolutionallayer.pdf and animation version can be found at
> > > http://cs231n.github.io/convolutional-networks/ for forward and
> backward
> > > of
> > > convolutional neuron.
> > >
> > > Thanks!
> > >
> > > --
> > > Best Regards, Edward J. Yoon
> > >
> > >
> > >
> > >
> >
> >
> > --
> > Zach Jaffee
> > B.S. Computer Science
> > Case Western Reserve University Class of 2017
> > Operations Director | WRUW FM 91.1 Cleveland
> > Secretary | Recruitment Chair | Phi Kappa Theta Fraternity
> > (917) 881-0646
> > zjaffee.com
> > github.com/ZJaffee
> >
> >
> >
>
>
> --
> Zach Jaffee
> B.S. Computer Science
> Case Western Reserve University Class of 2017
> Operations Director | WRUW FM 91.1 Cleveland
> Secretary | Recruitment Chair | Phi Kappa Theta Fraternity
> (917) 881-0646
> zjaffee.com
> github.com/ZJaffee
>
>
>


--
Zach Jaffee
B.S. Computer Science
Case Western Reserve University Class of 2017
Operations Director | WRUW FM 91.1 Cleveland
Secretary | Recruitment Chair | Phi Kappa Theta Fraternity
(917) 881-0646
zjaffee.com
github.com/ZJaffee