Mailing-List: contact dev-help@horn.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@horn.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <CABaiRmdLZBv--Pxu3=4Kd24-_D1pQbc9ahVWg2eKtH9Hm3NCNA@mail.gmail.com>
References: <01bd01d1ff32$764c3ad0$62e4b070$@samsung.com> <CACHLYnnZFA_e+v-50VHscAY8wgdp3hgA_a8yqEKbEq2W8JdE_g@mail.gmail.com>
 <CABaiRmfXVa4cPf+4xsqwV34H1AJV-v7sPG0tVTHiX7-zsGA4+A@mail.gmail.com>
 <CAGQgZQTpYPV9BjZaCWE-OO=Bb7x+2D_GGhtUeFsX54vZrNPNOw@mail.gmail.com>
 <CABaiRmf05zJnDEi1B0D4QMCVQ=zvBu1V1UR7T2Hn5+j02-LBbw@mail.gmail.com>
 <CAGQgZQQspdULmqv5T5hPHWN9-cFPLzCRa_yS65_DLOqKtB2dRg@mail.gmail.com>
 <CAGQgZQRPoS6ryhHY-YQuif58XUYLZ9jx-+pOSYwzqf8c6vYn6A@mail.gmail.com>
 <CABaiRme+HCJwdb6EnPBgbk1bEXQruQq=Fa+jZ+i=5PFHaPW5BQ@mail.gmail.com>
 <CGME20160906144154epcas1p3e17b9adfed2e3d0b0f84fa21065beffa@epcas1p3.samsung.com>
 <CABaiRmcWsxFRqsE+Lm6B7x8AUATJtDdBVOPUZbBAMonCSk1hxA@mail.gmail.com>
 <017f01d2088f$ca750d60$5f5f2820$@samsung.com> <CABaiRmdgt2J7F9mrKQpfv=4hM-XcOdHc8yB-Ga_cJo6kK0uBHg@mail.gmail.com>
 <018c01d20893$3fd564d0$bf802e70$@samsung.com> <CABaiRmdLZBv--Pxu3=4Kd24-_D1pQbc9ahVWg2eKtH9Hm3NCNA@mail.gmail.com>
From: Baran Topal <barantopal@barantopal.com>
Date: Mon, 31 Oct 2016 13:22:20 +0100
Message-ID: <CABaiRmfORaNZ5M6nW4_OzTUSj4mcC0aspihFLRoNcuN+xFXb1g@mail.gmail.com>
Subject: Re: Use vector instead of Iterable in neuron API
To: dev@horn.incubator.apache.org
Content-Type: multipart/alternative; boundary=047d7bae4520760f2005402842fe
archived-at: Mon, 31 Oct 2016 12:22:29 -0000

--047d7bae4520760f2005402842fe
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi Edward;

I was looking at this for a while but got stuck on this. I was changing the
code a lot so I think my approach might not be really correct.

For learning purposes, I appreciate if another developer can take the case
so that I can see the solution.

Br.

2016-09-07 14:10 GMT+02:00 Baran Topal <barantopal@barantopal.com>:

> Thanks Edward.
>
> I just cloned the repo and created the fork and update my fork.
>
> I created the following case,
>
> https://issues.apache.org/jira/browse/HORN-31
>
> Please assign me.
>
> Br.
>
> 2016-09-07 1:06 GMT+02:00 Edward J. Yoon <edward.yoon@samsung.com>:
> > If you're using Git, you're probably using pull requests. Here's info
> about
> > pull request:
> > https://cwiki.apache.org/confluence/display/HORN/How+To+Contribute
> >
> > And, please feel free to file this issue on JIRA:
> > http://issues.apache.org/jira/browse/HORN
> >
> > Then, I'll assign to you!
> >
> > --
> > Best Regards, Edward J. Yoon
> >
> >
> > -----Original Message-----
> > From: Baran Topal [mailto:barantopal@barantopal.com]
> > Sent: Wednesday, September 07, 2016 8:02 AM
> > To: dev@horn.incubator.apache.org
> > Subject: Re: Use vector instead of Iterable in neuron API
> >
> > Hi Edward,
> >
> > Many thanks. I will update this as soon as possible.
> >
> > Br.
> >
> > 7 Eylul 2016 Car?amba tarihinde, Edward J. Yoon <edward.yoon@samsung.co=
m
> >
> > yazd=C4=B1:
> >
> >> > directly into FloatVector and get rid of Synapse totally?
> >>
> >> Yes, I think that's more clear.
> >>
> >> Internally, each task processes neurons' computation on assigned data
> >> split.
> >> The LayeredNeuralNetwork.java contains everything, you can find the
> >> "n.forward(msg);" and "n.backward(msg);" at there. Before calling the
> >> forward() or backward() method, we can set the weight vector associate=
d
> >> with
> >> neuron using setWeightVector() method and set the argument value for t=
he
> >> forward() method like below:
> >>
> >> n.setWeightVector(weight vector e.g., weightMatrix.get(neuronID));
> >> n.forward(outputs from previous layer as a FloatVector);
> >>
> >> Then, user-side program can be done like below:
> >>
> >> forward(FloatVector input) {
> >>   this.getWeightVector(); // returns weight vector associated with
> itself.
> >>   float vectorSum;
> >>   for(float element : input) {
> >>     vectorSum +=3D element;
> >>   }
> >> }
> >>
> >> If you have any trouble, please don't hesitate ask here.
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >>
> >>
> >> -----Original Message-----
> >> From: Baran Topal [mailto:barantopal@barantopal.com <javascript:;>]
> >> Sent: Tuesday, September 06, 2016 11:42 PM
> >> To: dev@horn.incubator.apache.org <javascript:;>
> >> Subject: Re: Use vector instead of Iterable in neuron API
> >>
> >> Hi team and Edward;
> >>
> >> I have been checking this and got stuck how to convert the list
> >> structure to DenseFloatVector. Can you help on this?
> >>
> >> Let me explain:
> >>
> >> I saw that the concrete FloatVector is actually some sort of array
> >> structure which is not really compatible with Synapse and
> >> FloatWritables. Is the aim to convert Synapse construction logic
> >> directly into FloatVector and get rid of Synapse totally?
> >>
> >> //
> >>
> >> In the original code, we are passing a list with a structure of having
> >> Synapse and FloatWritables. I can see that a synapse can be
> >> constructed with a neuron id and 2 float writables.
> >>
> >> What I tried;
> >>
> >> 1) I added the following function in Neuron.java
> >>
> >>   public DenseFloatVector getWeightVector() {
> >>     DenseFloatVector dfv =3D new DenseFloatVector(getWeights());
> >>     return dfv;
> >>   }
> >>
> >> 2) I added the following function in NeuronInterface.java
> >>
> >> public void forward2(FloatVector messages) throws IOException;
> >>
> >> 3) I added the following function in TestNeuron.java
> >>
> >>     public void forward2(FloatVector messages) throws IOException {
> >>
> >>       long startTime =3D System.nanoTime();
> >>
> >>       float sum =3D messages.dot(this.getWeightVector());
> >>       this.feedforward(this.squashingFunction.apply(sum));
> >>
> >>       long endTime =3D System.nanoTime();
> >>
> >>       System.out.println("Execution time for the forward2 function is:
> >> " + ((endTime - startTime)));
> >>
> >>     }
> >> 4) I tried to refactor testProp() in TestNeuron.java with several ways
> >> but failed unfortunately with runtime error since weight is not having
> >> a value.
> >>
> >>   MyNeuron n_ =3D new MyNeuron();
> >>
> >>     FloatVector ds =3D new DenseFloatVector();
> >>
> >>     Iterator <Synapse<FloatWritable, FloatWritable>> li =3D x_.iterato=
r();
> >>     // Iterator<FloatVector.FloatVectorElement> ie =3D ds.iterate();
> >>
> >>     while(li.hasNext()) {
> >>       // ds.add(li.next());
> >>       // ie.
> >>       Synapse<FloatWritable, FloatWritable> ee =3D li.next();
> >>
> >>       ds.set(ee.getSenderID(), ee.getMessage());
> >>
> >>     }
> >>
> >>     float[] ff =3D new float[2];
> >>     ff[0] =3D 1.0f;
> >>     ff[1] =3D 0.5f;
> >>
> >>     float[] ffa =3D new float[2];
> >>     ffa[0] =3D 1.0f;
> >>     ffa[1] =3D 0.4f;
> >>
> >>     DenseFloatVector dss =3D new DenseFloatVector(ff);
> >>     DenseFloatVector dssa =3D new DenseFloatVector(ffa);
> >>
> >>     dss.add(dssa);
> >>
> >>
> >>     FloatWritable a =3D new FloatWritable(1.0f);
> >>     FloatWritable b =3D new FloatWritable(0.5f);
> >>
> >>     Synapse s =3D new Synapse(0, a, b);
> >>
> >>
> >>
> >>   //  dss.set(1, 0.5f);
> >>     //dss.set(1, 0.4f);
> >>
> >>    // DenseFloatVector ds =3D new DenseFloatVector(ff);
> >>
> >>     n_.forward2(dss); //forward2
> >>
> >>
> >>
> >> 2016-09-05 0:55 GMT+02:00 Baran Topal <barantopal@barantopal.com
> >> <javascript:;>>:
> >> > Hi;
> >> >
> >> > Thanks I am on it.
> >> >
> >> > Br.
> >> >
> >> > 2016-09-04 4:16 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org
> >> <javascript:;>>:
> >> >> P.S., so, if you want to test more, please see FloatVector and
> >> >> DenseFloatVector.
> >> >>
> >> >> On Sun, Sep 4, 2016 at 11:13 AM, Edward J. Yoon <
> edwardyoon@apache.org
> >> <javascript:;>>
> >> >> wrote:
> >> >>> Once we change the iterable input messages to the vector, we can
> >> >>> change the legacy code like below:
> >> >>>
> >> >>> public void forward(FloatVector input) {
> >> >>>   float sum =3D input.dot(this.getWeightVector());
> >> >>>   this.feedforward(this.squashingFunction.apply(sum));
> >> >>> }
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Sat, Sep 3, 2016 at 11:10 PM, Baran Topal <
> >> barantopal@barantopal.com <javascript:;>>
> >> >>> wrote:
> >> >>>> Sure.
> >> >>>>
> >> >>>> In the attached, TestNeuron.txt,
> >> >>>>
> >> >>>> 1) I put // baran as a comment for the added functions.
> >> >>>>
> >> >>>> 2) The added functions and created objects have _ as suffix
> >> >>>>
> >> >>>> (e.g. backward_)
> >> >>>>
> >> >>>>
> >> >>>> A correction: above test execution time values were via
> >> >>>> System.nanoTime().
> >> >>>>
> >> >>>> Br.
> >> >>>>
> >> >>>> 2016-09-03 14:05 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org
> >> <javascript:;>>:
> >> >>>>> Interesting. Can you share your test code?
> >> >>>>>
> >> >>>>> On Sat, Sep 3, 2016 at 2:17 AM, Baran Topal <
> >> barantopal@barantopal.com <javascript:;>>
> >> >>>>> wrote:
> >> >>>>>> Hi Edward and team;
> >> >>>>>>
> >> >>>>>> I had a brief test by refactoring Iterable to Vector and on
> >> >>>>>> TestNeuron.java, I can see some improved times. I didn't check
> for
> >> >>>>>> other existing test methods but it seems the execution times ar=
e
> >> >>>>>> improving for both forwarding and backwarding.
> >> >>>>>>
> >> >>>>>> These values are via System.currentTimeMillis().
> >> >>>>>>
> >> >>>>>> E.g.
> >> >>>>>>
> >> >>>>>>
> >> >>>>>> Execution time for the forward function is: 5722329
> >> >>>>>> Execution time for the backward function is: 31825
> >> >>>>>>
> >> >>>>>> Execution time for the refactored forward function is: 72330
> >> >>>>>> Execution time for the refactored backward function is: 4665
> >> >>>>>>
> >> >>>>>> Br.
> >> >>>>>>
> >> >>>>>> 2016-09-02 2:14 GMT+02:00 Yeonhee Lee <ssallys0130@gmail.com
> >> <javascript:;>>:
> >> >>>>>>> Hi Edward,
> >> >>>>>>>
> >> >>>>>>> If we don't have that kind of method in the neuron, I guess it=
's
> >> >>>>>>> appropriate to put the method to the neuron.
> >> >>>>>>> That can be one of the distinct features of Horn.
> >> >>>>>>>
> >> >>>>>>> Regards,
> >> >>>>>>> Yeonhee
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> 2016-08-26 9:40 GMT+09:00 Edward J. Yoon <
> edward.yoon@samsung.com
> >> <javascript:;>>:
> >> >>>>>>>
> >> >>>>>>>> Hi forks,
> >> >>>>>>>>
> >> >>>>>>>> Our current neuron API is designed like:
> >> >>>>>>>> https://github.com/apache/incubator-horn/blob/master/
> >> >>>>>>>> README.md#programming-m
> >> >>>>>>>> odel
> >> >>>>>>>>
> >> >>>>>>>> In forward() method, each neuron receives the pairs of the
> inputs
> >> x1,
> >> >>>>>>>> x2,
> >> >>>>>>>> ... xn from other neurons and weights w1, w2, ... wn like
> below:
> >> >>>>>>>>
> >> >>>>>>>>   public void forward(Iterable<M> messages) throws IOExceptio=
n;
> >> >>>>>>>>
> >> >>>>>>>> Instead of this, I suggest that we use just vector like below=
:
> >> >>>>>>>>
> >> >>>>>>>>   /**
> >> >>>>>>>>    * @param input vector from other neurons
> >> >>>>>>>>    * /
> >> >>>>>>>>   public void forward(Vector input) throws IOException;
> >> >>>>>>>>
> >> >>>>>>>> And, the neuron provides a getWeightVector() method that
> returns
> >> >>>>>>>> weight
> >> >>>>>>>> vector associated with itself. I think this is more make sens=
e
> >> than
> >> >>>>>>>> current
> >> >>>>>>>> version, and more easy to use GPU in the future.
> >> >>>>>>>>
> >> >>>>>>>> What do you think?
> >> >>>>>>>>
> >> >>>>>>>> Thanks.
> >> >>>>>>>>
> >> >>>>>>>> --
> >> >>>>>>>> Best Regards, Edward J. Yoon
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>> --
> >> >>>>> Best Regards, Edward J. Yoon
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Best Regards, Edward J. Yoon
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards, Edward J. Yoon
> >>
> >>
> >>
> >>
> >
> >
>

--047d7bae4520760f2005402842fe--