Mailing-List: contact dev-help@horn.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@horn.incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <018c01d20893$3fd564d0$bf802e70$@samsung.com>
References: <01bd01d1ff32$764c3ad0$62e4b070$@samsung.com> <CACHLYnnZFA_e+v-50VHscAY8wgdp3hgA_a8yqEKbEq2W8JdE_g@mail.gmail.com>
 <CABaiRmfXVa4cPf+4xsqwV34H1AJV-v7sPG0tVTHiX7-zsGA4+A@mail.gmail.com>
 <CAGQgZQTpYPV9BjZaCWE-OO=Bb7x+2D_GGhtUeFsX54vZrNPNOw@mail.gmail.com>
 <CABaiRmf05zJnDEi1B0D4QMCVQ=zvBu1V1UR7T2Hn5+j02-LBbw@mail.gmail.com>
 <CAGQgZQQspdULmqv5T5hPHWN9-cFPLzCRa_yS65_DLOqKtB2dRg@mail.gmail.com>
 <CAGQgZQRPoS6ryhHY-YQuif58XUYLZ9jx-+pOSYwzqf8c6vYn6A@mail.gmail.com>
 <CABaiRme+HCJwdb6EnPBgbk1bEXQruQq=Fa+jZ+i=5PFHaPW5BQ@mail.gmail.com>
 <CGME20160906144154epcas1p3e17b9adfed2e3d0b0f84fa21065beffa@epcas1p3.samsung.com>
 <CABaiRmcWsxFRqsE+Lm6B7x8AUATJtDdBVOPUZbBAMonCSk1hxA@mail.gmail.com>
 <017f01d2088f$ca750d60$5f5f2820$@samsung.com> <CABaiRmdgt2J7F9mrKQpfv=4hM-XcOdHc8yB-Ga_cJo6kK0uBHg@mail.gmail.com>
 <018c01d20893$3fd564d0$bf802e70$@samsung.com>
From: Baran Topal <barantopal@barantopal.com>
Date: Wed, 7 Sep 2016 14:10:54 +0200
Message-ID: <CABaiRmdLZBv--Pxu3=4Kd24-_D1pQbc9ahVWg2eKtH9Hm3NCNA@mail.gmail.com>
Subject: Re: Use vector instead of Iterable in neuron API
To: dev@horn.incubator.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
archived-at: Wed, 07 Sep 2016 12:11:07 -0000

Thanks Edward.

I just cloned the repo and created the fork and update my fork.

I created the following case,

https://issues.apache.org/jira/browse/HORN-31

Please assign me.

Br.

2016-09-07 1:06 GMT+02:00 Edward J. Yoon <edward.yoon@samsung.com>:
> If you're using Git, you're probably using pull requests. Here's info abo=
ut
> pull request:
> https://cwiki.apache.org/confluence/display/HORN/How+To+Contribute
>
> And, please feel free to file this issue on JIRA:
> http://issues.apache.org/jira/browse/HORN
>
> Then, I'll assign to you!
>
> --
> Best Regards, Edward J. Yoon
>
>
> -----Original Message-----
> From: Baran Topal [mailto:barantopal@barantopal.com]
> Sent: Wednesday, September 07, 2016 8:02 AM
> To: dev@horn.incubator.apache.org
> Subject: Re: Use vector instead of Iterable in neuron API
>
> Hi Edward,
>
> Many thanks. I will update this as soon as possible.
>
> Br.
>
> 7 Eylul 2016 Car?amba tarihinde, Edward J. Yoon <edward.yoon@samsung.com>
> yazd=C4=B1:
>
>> > directly into FloatVector and get rid of Synapse totally?
>>
>> Yes, I think that's more clear.
>>
>> Internally, each task processes neurons' computation on assigned data
>> split.
>> The LayeredNeuralNetwork.java contains everything, you can find the
>> "n.forward(msg);" and "n.backward(msg);" at there. Before calling the
>> forward() or backward() method, we can set the weight vector associated
>> with
>> neuron using setWeightVector() method and set the argument value for the
>> forward() method like below:
>>
>> n.setWeightVector(weight vector e.g., weightMatrix.get(neuronID));
>> n.forward(outputs from previous layer as a FloatVector);
>>
>> Then, user-side program can be done like below:
>>
>> forward(FloatVector input) {
>>   this.getWeightVector(); // returns weight vector associated with itsel=
f.
>>   float vectorSum;
>>   for(float element : input) {
>>     vectorSum +=3D element;
>>   }
>> }
>>
>> If you have any trouble, please don't hesitate ask here.
>>
>> --
>> Best Regards, Edward J. Yoon
>>
>>
>> -----Original Message-----
>> From: Baran Topal [mailto:barantopal@barantopal.com <javascript:;>]
>> Sent: Tuesday, September 06, 2016 11:42 PM
>> To: dev@horn.incubator.apache.org <javascript:;>
>> Subject: Re: Use vector instead of Iterable in neuron API
>>
>> Hi team and Edward;
>>
>> I have been checking this and got stuck how to convert the list
>> structure to DenseFloatVector. Can you help on this?
>>
>> Let me explain:
>>
>> I saw that the concrete FloatVector is actually some sort of array
>> structure which is not really compatible with Synapse and
>> FloatWritables. Is the aim to convert Synapse construction logic
>> directly into FloatVector and get rid of Synapse totally?
>>
>> //
>>
>> In the original code, we are passing a list with a structure of having
>> Synapse and FloatWritables. I can see that a synapse can be
>> constructed with a neuron id and 2 float writables.
>>
>> What I tried;
>>
>> 1) I added the following function in Neuron.java
>>
>>   public DenseFloatVector getWeightVector() {
>>     DenseFloatVector dfv =3D new DenseFloatVector(getWeights());
>>     return dfv;
>>   }
>>
>> 2) I added the following function in NeuronInterface.java
>>
>> public void forward2(FloatVector messages) throws IOException;
>>
>> 3) I added the following function in TestNeuron.java
>>
>>     public void forward2(FloatVector messages) throws IOException {
>>
>>       long startTime =3D System.nanoTime();
>>
>>       float sum =3D messages.dot(this.getWeightVector());
>>       this.feedforward(this.squashingFunction.apply(sum));
>>
>>       long endTime =3D System.nanoTime();
>>
>>       System.out.println("Execution time for the forward2 function is:
>> " + ((endTime - startTime)));
>>
>>     }
>> 4) I tried to refactor testProp() in TestNeuron.java with several ways
>> but failed unfortunately with runtime error since weight is not having
>> a value.
>>
>>   MyNeuron n_ =3D new MyNeuron();
>>
>>     FloatVector ds =3D new DenseFloatVector();
>>
>>     Iterator <Synapse<FloatWritable, FloatWritable>> li =3D x_.iterator(=
);
>>     // Iterator<FloatVector.FloatVectorElement> ie =3D ds.iterate();
>>
>>     while(li.hasNext()) {
>>       // ds.add(li.next());
>>       // ie.
>>       Synapse<FloatWritable, FloatWritable> ee =3D li.next();
>>
>>       ds.set(ee.getSenderID(), ee.getMessage());
>>
>>     }
>>
>>     float[] ff =3D new float[2];
>>     ff[0] =3D 1.0f;
>>     ff[1] =3D 0.5f;
>>
>>     float[] ffa =3D new float[2];
>>     ffa[0] =3D 1.0f;
>>     ffa[1] =3D 0.4f;
>>
>>     DenseFloatVector dss =3D new DenseFloatVector(ff);
>>     DenseFloatVector dssa =3D new DenseFloatVector(ffa);
>>
>>     dss.add(dssa);
>>
>>
>>     FloatWritable a =3D new FloatWritable(1.0f);
>>     FloatWritable b =3D new FloatWritable(0.5f);
>>
>>     Synapse s =3D new Synapse(0, a, b);
>>
>>
>>
>>   //  dss.set(1, 0.5f);
>>     //dss.set(1, 0.4f);
>>
>>    // DenseFloatVector ds =3D new DenseFloatVector(ff);
>>
>>     n_.forward2(dss); //forward2
>>
>>
>>
>> 2016-09-05 0:55 GMT+02:00 Baran Topal <barantopal@barantopal.com
>> <javascript:;>>:
>> > Hi;
>> >
>> > Thanks I am on it.
>> >
>> > Br.
>> >
>> > 2016-09-04 4:16 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org
>> <javascript:;>>:
>> >> P.S., so, if you want to test more, please see FloatVector and
>> >> DenseFloatVector.
>> >>
>> >> On Sun, Sep 4, 2016 at 11:13 AM, Edward J. Yoon <edwardyoon@apache.or=
g
>> <javascript:;>>
>> >> wrote:
>> >>> Once we change the iterable input messages to the vector, we can
>> >>> change the legacy code like below:
>> >>>
>> >>> public void forward(FloatVector input) {
>> >>>   float sum =3D input.dot(this.getWeightVector());
>> >>>   this.feedforward(this.squashingFunction.apply(sum));
>> >>> }
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On Sat, Sep 3, 2016 at 11:10 PM, Baran Topal <
>> barantopal@barantopal.com <javascript:;>>
>> >>> wrote:
>> >>>> Sure.
>> >>>>
>> >>>> In the attached, TestNeuron.txt,
>> >>>>
>> >>>> 1) I put // baran as a comment for the added functions.
>> >>>>
>> >>>> 2) The added functions and created objects have _ as suffix
>> >>>>
>> >>>> (e.g. backward_)
>> >>>>
>> >>>>
>> >>>> A correction: above test execution time values were via
>> >>>> System.nanoTime().
>> >>>>
>> >>>> Br.
>> >>>>
>> >>>> 2016-09-03 14:05 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org
>> <javascript:;>>:
>> >>>>> Interesting. Can you share your test code?
>> >>>>>
>> >>>>> On Sat, Sep 3, 2016 at 2:17 AM, Baran Topal <
>> barantopal@barantopal.com <javascript:;>>
>> >>>>> wrote:
>> >>>>>> Hi Edward and team;
>> >>>>>>
>> >>>>>> I had a brief test by refactoring Iterable to Vector and on
>> >>>>>> TestNeuron.java, I can see some improved times. I didn't check fo=
r
>> >>>>>> other existing test methods but it seems the execution times are
>> >>>>>> improving for both forwarding and backwarding.
>> >>>>>>
>> >>>>>> These values are via System.currentTimeMillis().
>> >>>>>>
>> >>>>>> E.g.
>> >>>>>>
>> >>>>>>
>> >>>>>> Execution time for the forward function is: 5722329
>> >>>>>> Execution time for the backward function is: 31825
>> >>>>>>
>> >>>>>> Execution time for the refactored forward function is: 72330
>> >>>>>> Execution time for the refactored backward function is: 4665
>> >>>>>>
>> >>>>>> Br.
>> >>>>>>
>> >>>>>> 2016-09-02 2:14 GMT+02:00 Yeonhee Lee <ssallys0130@gmail.com
>> <javascript:;>>:
>> >>>>>>> Hi Edward,
>> >>>>>>>
>> >>>>>>> If we don't have that kind of method in the neuron, I guess it's
>> >>>>>>> appropriate to put the method to the neuron.
>> >>>>>>> That can be one of the distinct features of Horn.
>> >>>>>>>
>> >>>>>>> Regards,
>> >>>>>>> Yeonhee
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> 2016-08-26 9:40 GMT+09:00 Edward J. Yoon <edward.yoon@samsung.co=
m
>> <javascript:;>>:
>> >>>>>>>
>> >>>>>>>> Hi forks,
>> >>>>>>>>
>> >>>>>>>> Our current neuron API is designed like:
>> >>>>>>>> https://github.com/apache/incubator-horn/blob/master/
>> >>>>>>>> README.md#programming-m
>> >>>>>>>> odel
>> >>>>>>>>
>> >>>>>>>> In forward() method, each neuron receives the pairs of the inpu=
ts
>> x1,
>> >>>>>>>> x2,
>> >>>>>>>> ... xn from other neurons and weights w1, w2, ... wn like below=
:
>> >>>>>>>>
>> >>>>>>>>   public void forward(Iterable<M> messages) throws IOException;
>> >>>>>>>>
>> >>>>>>>> Instead of this, I suggest that we use just vector like below:
>> >>>>>>>>
>> >>>>>>>>   /**
>> >>>>>>>>    * @param input vector from other neurons
>> >>>>>>>>    * /
>> >>>>>>>>   public void forward(Vector input) throws IOException;
>> >>>>>>>>
>> >>>>>>>> And, the neuron provides a getWeightVector() method that return=
s
>> >>>>>>>> weight
>> >>>>>>>> vector associated with itself. I think this is more make sense
>> than
>> >>>>>>>> current
>> >>>>>>>> version, and more easy to use GPU in the future.
>> >>>>>>>>
>> >>>>>>>> What do you think?
>> >>>>>>>>
>> >>>>>>>> Thanks.
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> Best Regards, Edward J. Yoon
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Best Regards, Edward J. Yoon
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Best Regards, Edward J. Yoon
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards, Edward J. Yoon
>>
>>
>>
>>
>
>