horn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Baran Topal <baranto...@barantopal.com>
Subject Re: Use vector instead of Iterable in neuron API
Date Thu, 03 Nov 2016 22:10:22 GMT
Hi Edward,

Thanks. I will.

Br.

1 Kasım 2016 Salı tarihinde, Edward J. Yoon <edward.yoon@samsung.com> yazdı:

> Sure, I'll help on it. If you can try to change the user-side code as we
> discussed, I think I can change the internal others.
>
> --
> Best Regards, Edward J. Yoon
>
>
> -----Original Message-----
> From: Baran Topal [mailto:barantopal@barantopal.com <javascript:;>]
> Sent: Monday, October 31, 2016 9:22 PM
> To: dev@horn.incubator.apache.org <javascript:;>
> Subject: Re: Use vector instead of Iterable in neuron API
>
> Hi Edward;
>
> I was looking at this for a while but got stuck on this. I was changing the
> code a lot so I think my approach might not be really correct.
>
> For learning purposes, I appreciate if another developer can take the case
> so that I can see the solution.
>
> Br.
>
> 2016-09-07 14:10 GMT+02:00 Baran Topal <barantopal@barantopal.com
> <javascript:;>>:
>
> > Thanks Edward.
> >
> > I just cloned the repo and created the fork and update my fork.
> >
> > I created the following case,
> >
> > https://issues.apache.org/jira/browse/HORN-31
> >
> > Please assign me.
> >
> > Br.
> >
> > 2016-09-07 1:06 GMT+02:00 Edward J. Yoon <edward.yoon@samsung.com
> <javascript:;>>:
> > > If you're using Git, you're probably using pull requests. Here's info
> > about
> > > pull request:
> > > https://cwiki.apache.org/confluence/display/HORN/How+To+Contribute
> > >
> > > And, please feel free to file this issue on JIRA:
> > > http://issues.apache.org/jira/browse/HORN
> > >
> > > Then, I'll assign to you!
> > >
> > > --
> > > Best Regards, Edward J. Yoon
> > >
> > >
> > > -----Original Message-----
> > > From: Baran Topal [mailto:barantopal@barantopal.com <javascript:;>]
> > > Sent: Wednesday, September 07, 2016 8:02 AM
> > > To: dev@horn.incubator.apache.org <javascript:;>
> > > Subject: Re: Use vector instead of Iterable in neuron API
> > >
> > > Hi Edward,
> > >
> > > Many thanks. I will update this as soon as possible.
> > >
> > > Br.
> > >
> > > 7 Eylul 2016 Car?amba tarihinde, Edward J. Yoon <
> edward.yoon@samsung.com <javascript:;>
> > >
> > > yazdı:
> > >
> > >> > directly into FloatVector and get rid of Synapse totally?
> > >>
> > >> Yes, I think that's more clear.
> > >>
> > >> Internally, each task processes neurons' computation on assigned data
> > >> split.
> > >> The LayeredNeuralNetwork.java contains everything, you can find the
> > >> "n.forward(msg);" and "n.backward(msg);" at there. Before calling the
> > >> forward() or backward() method, we can set the weight vector
> associated
> > >> with
> > >> neuron using setWeightVector() method and set the argument value for
> the
> > >> forward() method like below:
> > >>
> > >> n.setWeightVector(weight vector e.g., weightMatrix.get(neuronID));
> > >> n.forward(outputs from previous layer as a FloatVector);
> > >>
> > >> Then, user-side program can be done like below:
> > >>
> > >> forward(FloatVector input) {
> > >>   this.getWeightVector(); // returns weight vector associated with
> > itself.>
> >  >>   float vectorSum;
> > >>   for(float element : input) {
> > >>     vectorSum += element;
> > >>   }
> > >> }
> > >>
> > >> If you have any trouble, please don't hesitate ask here.
> > >>
> > >> --
> > >> Best Regards, Edward J. Yoon
> > >>
> > >>
> > >> -----Original Message-----
> > >> From: Baran Topal [mailto:barantopal@barantopal.com <javascript:;>
> <javascript:;>]
> > >> Sent: Tuesday, September 06, 2016 11:42 PM
> > >> To: dev@horn.incubator.apache.org <javascript:;> <javascript:;>
> > >> Subject: Re: Use vector instead of Iterable in neuron API
> > >>
> > >> Hi team and Edward;
> > >>
> > >> I have been checking this and got stuck how to convert the list
> > >> structure to DenseFloatVector. Can you help on this?
> > >>
> > >> Let me explain:
> > >>
> > >> I saw that the concrete FloatVector is actually some sort of array
> > >> structure which is not really compatible with Synapse and
> > >> FloatWritables. Is the aim to convert Synapse construction logic
> > >> directly into FloatVector and get rid of Synapse totally?
> > >>
> > >> //
> > >>
> > >> In the original code, we are passing a list with a structure of having
> > >> Synapse and FloatWritables. I can see that a synapse can be
> > >> constructed with a neuron id and 2 float writables.
> > >>
> > >> What I tried;
> > >>
> > >> 1) I added the following function in Neuron.java
> > >>
> > >>   public DenseFloatVector getWeightVector() {
> > >>     DenseFloatVector dfv = new DenseFloatVector(getWeights());
> > >>     return dfv;
> > >>   }
> > >>
> > >> 2) I added the following function in NeuronInterface.java
> > >>
> > >> public void forward2(FloatVector messages) throws IOException;
> > >>
> > >> 3) I added the following function in TestNeuron.java
> > >>
> > >>     public void forward2(FloatVector messages) throws IOException {
> > >>
> > >>       long startTime = System.nanoTime();
> > >>
> > >>       float sum = messages.dot(this.getWeightVector());
> > >>       this.feedforward(this.squashingFunction.apply(sum));
> > >>
> > >>       long endTime = System.nanoTime();
> > >>
> > >>       System.out.println("Execution time for the forward2 function is:
> > >> " + ((endTime - startTime)));
> > >>
> > >>     }
> > >> 4) I tried to refactor testProp() in TestNeuron.java with several ways
> > >> but failed unfortunately with runtime error since weight is not having
> > >> a value.
> > >>
> > >>   MyNeuron n_ = new MyNeuron();
> > >>
> > >>     FloatVector ds = new DenseFloatVector();
> > >>
> > >>     Iterator <Synapse<FloatWritable, FloatWritable>> li =
> x_.iterator();
> > >>     // Iterator<FloatVector.FloatVectorElement> ie = ds.iterate();
> > >>
> > >>     while(li.hasNext()) {
> > >>       // ds.add(li.next());
> > >>       // ie.
> > >>       Synapse<FloatWritable, FloatWritable> ee = li.next();
> > >>
> > >>       ds.set(ee.getSenderID(), ee.getMessage());
> > >>
> > >>     }
> > >>
> > >>     float[] ff = new float[2];
> > >>     ff[0] = 1.0f;
> > >>     ff[1] = 0.5f;
> > >>
> > >>     float[] ffa = new float[2];
> > >>     ffa[0] = 1.0f;
> > >>     ffa[1] = 0.4f;
> > >>
> > >>     DenseFloatVector dss = new DenseFloatVector(ff);
> > >>     DenseFloatVector dssa = new DenseFloatVector(ffa);
> > >>
> > >>     dss.add(dssa);
> > >>
> > >>
> > >>     FloatWritable a = new FloatWritable(1.0f);
> > >>     FloatWritable b = new FloatWritable(0.5f);
> > >>
> > >>     Synapse s = new Synapse(0, a, b);
> > >>
> > >>
> > >>
> > >>   //  dss.set(1, 0.5f);
> > >>     //dss.set(1, 0.4f);
> > >>
> > >>    // DenseFloatVector ds = new DenseFloatVector(ff);
> > >>
> > >>     n_.forward2(dss); //forward2
> > >>
> > >>
> > >>
> > >> 2016-09-05 0:55 GMT+02:00 Baran Topal <barantopal@barantopal.com
> <javascript:;>
> > >> <javascript:;>>:
> > >> > Hi;
> > >> >
> > >> > Thanks I am on it.
> > >> >
> > >> > Br.
> > >> >
> > >> > 2016-09-04 4:16 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org
> <javascript:;>
> > >> <javascript:;>>:
> > >> >> P.S., so, if you want to test more, please see FloatVector and
> > >> >> DenseFloatVector.
> > >> >>
> > >> >> On Sun, Sep 4, 2016 at 11:13 AM, Edward J. Yoon <
> > edwardyoon@apache.org <javascript:;>
> > >> <javascript:;>>
> > >> >> wrote:
> > >> >>> Once we change the iterable input messages to the vector,
we can
> > >> >>> change the legacy code like below:
> > >> >>>
> > >> >>> public void forward(FloatVector input) {
> > >> >>>   float sum = input.dot(this.getWeightVector());
> > >> >>>   this.feedforward(this.squashingFunction.apply(sum));
> > >> >>> }
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>> On Sat, Sep 3, 2016 at 11:10 PM, Baran Topal <
> > >> barantopal@barantopal.com <javascript:;> <javascript:;>>
> > >> >>> wrote:
> > >> >>>> Sure.
> > >> >>>>
> > >> >>>> In the attached, TestNeuron.txt,
> > >> >>>>
> > >> >>>> 1) I put // baran as a comment for the added functions.
> > >> >>>>
> > >> >>>> 2) The added functions and created objects have _ as suffix
> > >> >>>>
> > >> >>>> (e.g. backward_)
> > >> >>>>
> > >> >>>>
> > >> >>>> A correction: above test execution time values were via
> > >> >>>> System.nanoTime().
> > >> >>>>
> > >> >>>> Br.
> > >> >>>>
> > >> >>>> 2016-09-03 14:05 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org
> <javascript:;>
> > >> <javascript:;>>:
> > >> >>>>> Interesting. Can you share your test code?
> > >> >>>>>
> > >> >>>>> On Sat, Sep 3, 2016 at 2:17 AM, Baran Topal <
> > >> barantopal@barantopal.com <javascript:;> <javascript:;>>
> > >> >>>>> wrote:
> > >> >>>>>> Hi Edward and team;
> > >> >>>>>>
> > >> >>>>>> I had a brief test by refactoring Iterable to
Vector and on
> > >> >>>>>> TestNeuron.java, I can see some improved times.
I didn't check
> > for
> > >> >>>>>> other existing test methods but it seems the execution
times
> are
> > >> >>>>>> improving for both forwarding and backwarding.
> > >> >>>>>>
> > >> >>>>>> These values are via System.currentTimeMillis().
> > >> >>>>>>
> > >> >>>>>> E.g.
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>> Execution time for the forward function is: 5722329
> > >> >>>>>> Execution time for the backward function is: 31825
> > >> >>>>>>
> > >> >>>>>> Execution time for the refactored forward function
is: 72330
> > >> >>>>>> Execution time for the refactored backward function
is: 4665
> > >> >>>>>>
> > >> >>>>>> Br.
> > >> >>>>>>
> > >> >>>>>> 2016-09-02 2:14 GMT+02:00 Yeonhee Lee <ssallys0130@gmail.com
> <javascript:;>
> > >> <javascript:;>>:
> > >> >>>>>>> Hi Edward,
> > >> >>>>>>>
> > >> >>>>>>> If we don't have that kind of method in the
neuron, I guess
> it's
> > >> >>>>>>> appropriate to put the method to the neuron.
> > >> >>>>>>> That can be one of the distinct features of
Horn.
> > >> >>>>>>>
> > >> >>>>>>> Regards,
> > >> >>>>>>> Yeonhee
> > >> >>>>>>>
> > >> >>>>>>>
> > >> >>>>>>> 2016-08-26 9:40 GMT+09:00 Edward J. Yoon <
> > edward.yoon@samsung.com <javascript:;>
> > >> <javascript:;>>:
> > >> >>>>>>>
> > >> >>>>>>>> Hi forks,
> > >> >>>>>>>>
> > >> >>>>>>>> Our current neuron API is designed like:
> > >> >>>>>>>> https://github.com/apache/incubator-horn/blob/master/
> > >> >>>>>>>> README.md#programming-m
> > >> >>>>>>>> odel
> > >> >>>>>>>>
> > >> >>>>>>>> In forward() method, each neuron receives
the pairs of the
> > inputs
> > >> x1,
> > >> >>>>>>>> x2,
> > >> >>>>>>>> ... xn from other neurons and weights
w1, w2, ... wn like
> > below:
> > >> >>>>>>>>
> > >> >>>>>>>>   public void forward(Iterable<M>
messages) throws
> IOException;
> > >> >>>>>>>>
> > >> >>>>>>>> Instead of this, I suggest that we use
just vector like
> below:
> > >> >>>>>>>>
> > >> >>>>>>>>   /**
> > >> >>>>>>>>    * @param input vector from other neurons
> > >> >>>>>>>>    * /
> > >> >>>>>>>>   public void forward(Vector input) throws
IOException;
> > >> >>>>>>>>
> > >> >>>>>>>> And, the neuron provides a getWeightVector()
method that
> > returns
> > >> >>>>>>>> weight
> > >> >>>>>>>> vector associated with itself. I think
this is more make
> sense
> > >> than
> > >> >>>>>>>> current
> > >> >>>>>>>> version, and more easy to use GPU in the
future.
> > >> >>>>>>>>
> > >> >>>>>>>> What do you think?
> > >> >>>>>>>>
> > >> >>>>>>>> Thanks.
> > >> >>>>>>>>
> > >> >>>>>>>> --
> > >> >>>>>>>> Best Regards, Edward J. Yoon
> > >> >>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>>>>
> > >> >>>>>
> > >> >>>>>
> > >> >>>>>
> > >> >>>>> --
> > >> >>>>> Best Regards, Edward J. Yoon
> > >> >>>
> > >> >>>
> > >> >>>
> > >> >>> --
> > >> >>> Best Regards, Edward J. Yoon
> > >> >>
> > >> >>
> > >> >>
> > >> >> --
> > >> >> Best Regards, Edward J. Yoon
> > >>
> > >>
> > >>
> > >>
> > >
> > >
> >
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message