horn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edward.y...@samsung.com>
Subject RE: Use vector instead of Iterable in neuron API
Date Mon, 31 Oct 2016 23:54:28 GMT
Sure, I'll help on it. If you can try to change the user-side code as we 
discussed, I think I can change the internal others.

--
Best Regards, Edward J. Yoon


-----Original Message-----
From: Baran Topal [mailto:barantopal@barantopal.com]
Sent: Monday, October 31, 2016 9:22 PM
To: dev@horn.incubator.apache.org
Subject: Re: Use vector instead of Iterable in neuron API

Hi Edward;

I was looking at this for a while but got stuck on this. I was changing the
code a lot so I think my approach might not be really correct.

For learning purposes, I appreciate if another developer can take the case
so that I can see the solution.

Br.

2016-09-07 14:10 GMT+02:00 Baran Topal <barantopal@barantopal.com>:

> Thanks Edward.
>
> I just cloned the repo and created the fork and update my fork.
>
> I created the following case,
>
> https://issues.apache.org/jira/browse/HORN-31
>
> Please assign me.
>
> Br.
>
> 2016-09-07 1:06 GMT+02:00 Edward J. Yoon <edward.yoon@samsung.com>:
> > If you're using Git, you're probably using pull requests. Here's info
> about
> > pull request:
> > https://cwiki.apache.org/confluence/display/HORN/How+To+Contribute
> >
> > And, please feel free to file this issue on JIRA:
> > http://issues.apache.org/jira/browse/HORN
> >
> > Then, I'll assign to you!
> >
> > --
> > Best Regards, Edward J. Yoon
> >
> >
> > -----Original Message-----
> > From: Baran Topal [mailto:barantopal@barantopal.com]
> > Sent: Wednesday, September 07, 2016 8:02 AM
> > To: dev@horn.incubator.apache.org
> > Subject: Re: Use vector instead of Iterable in neuron API
> >
> > Hi Edward,
> >
> > Many thanks. I will update this as soon as possible.
> >
> > Br.
> >
> > 7 Eylul 2016 Car?amba tarihinde, Edward J. Yoon <edward.yoon@samsung.com
> >
> > yazd─▒:
> >
> >> > directly into FloatVector and get rid of Synapse totally?
> >>
> >> Yes, I think that's more clear.
> >>
> >> Internally, each task processes neurons' computation on assigned data
> >> split.
> >> The LayeredNeuralNetwork.java contains everything, you can find the
> >> "n.forward(msg);" and "n.backward(msg);" at there. Before calling the
> >> forward() or backward() method, we can set the weight vector associated
> >> with
> >> neuron using setWeightVector() method and set the argument value for the
> >> forward() method like below:
> >>
> >> n.setWeightVector(weight vector e.g., weightMatrix.get(neuronID));
> >> n.forward(outputs from previous layer as a FloatVector);
> >>
> >> Then, user-side program can be done like below:
> >>
> >> forward(FloatVector input) {
> >>   this.getWeightVector(); // returns weight vector associated with
> itself.> 
>  >>   float vectorSum;
> >>   for(float element : input) {
> >>     vectorSum += element;
> >>   }
> >> }
> >>
> >> If you have any trouble, please don't hesitate ask here.
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >>
> >>
> >> -----Original Message-----
> >> From: Baran Topal [mailto:barantopal@barantopal.com <javascript:;>]
> >> Sent: Tuesday, September 06, 2016 11:42 PM
> >> To: dev@horn.incubator.apache.org <javascript:;>
> >> Subject: Re: Use vector instead of Iterable in neuron API
> >>
> >> Hi team and Edward;
> >>
> >> I have been checking this and got stuck how to convert the list
> >> structure to DenseFloatVector. Can you help on this?
> >>
> >> Let me explain:
> >>
> >> I saw that the concrete FloatVector is actually some sort of array
> >> structure which is not really compatible with Synapse and
> >> FloatWritables. Is the aim to convert Synapse construction logic
> >> directly into FloatVector and get rid of Synapse totally?
> >>
> >> //
> >>
> >> In the original code, we are passing a list with a structure of having
> >> Synapse and FloatWritables. I can see that a synapse can be
> >> constructed with a neuron id and 2 float writables.
> >>
> >> What I tried;
> >>
> >> 1) I added the following function in Neuron.java
> >>
> >>   public DenseFloatVector getWeightVector() {
> >>     DenseFloatVector dfv = new DenseFloatVector(getWeights());
> >>     return dfv;
> >>   }
> >>
> >> 2) I added the following function in NeuronInterface.java
> >>
> >> public void forward2(FloatVector messages) throws IOException;
> >>
> >> 3) I added the following function in TestNeuron.java
> >>
> >>     public void forward2(FloatVector messages) throws IOException {
> >>
> >>       long startTime = System.nanoTime();
> >>
> >>       float sum = messages.dot(this.getWeightVector());
> >>       this.feedforward(this.squashingFunction.apply(sum));
> >>
> >>       long endTime = System.nanoTime();
> >>
> >>       System.out.println("Execution time for the forward2 function is:
> >> " + ((endTime - startTime)));
> >>
> >>     }
> >> 4) I tried to refactor testProp() in TestNeuron.java with several ways
> >> but failed unfortunately with runtime error since weight is not having
> >> a value.
> >>
> >>   MyNeuron n_ = new MyNeuron();
> >>
> >>     FloatVector ds = new DenseFloatVector();
> >>
> >>     Iterator <Synapse<FloatWritable, FloatWritable>> li = x_.iterator();
> >>     // Iterator<FloatVector.FloatVectorElement> ie = ds.iterate();
> >>
> >>     while(li.hasNext()) {
> >>       // ds.add(li.next());
> >>       // ie.
> >>       Synapse<FloatWritable, FloatWritable> ee = li.next();
> >>
> >>       ds.set(ee.getSenderID(), ee.getMessage());
> >>
> >>     }
> >>
> >>     float[] ff = new float[2];
> >>     ff[0] = 1.0f;
> >>     ff[1] = 0.5f;
> >>
> >>     float[] ffa = new float[2];
> >>     ffa[0] = 1.0f;
> >>     ffa[1] = 0.4f;
> >>
> >>     DenseFloatVector dss = new DenseFloatVector(ff);
> >>     DenseFloatVector dssa = new DenseFloatVector(ffa);
> >>
> >>     dss.add(dssa);
> >>
> >>
> >>     FloatWritable a = new FloatWritable(1.0f);
> >>     FloatWritable b = new FloatWritable(0.5f);
> >>
> >>     Synapse s = new Synapse(0, a, b);
> >>
> >>
> >>
> >>   //  dss.set(1, 0.5f);
> >>     //dss.set(1, 0.4f);
> >>
> >>    // DenseFloatVector ds = new DenseFloatVector(ff);
> >>
> >>     n_.forward2(dss); //forward2
> >>
> >>
> >>
> >> 2016-09-05 0:55 GMT+02:00 Baran Topal <barantopal@barantopal.com
> >> <javascript:;>>:
> >> > Hi;
> >> >
> >> > Thanks I am on it.
> >> >
> >> > Br.
> >> >
> >> > 2016-09-04 4:16 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org
> >> <javascript:;>>:
> >> >> P.S., so, if you want to test more, please see FloatVector and
> >> >> DenseFloatVector.
> >> >>
> >> >> On Sun, Sep 4, 2016 at 11:13 AM, Edward J. Yoon <
> edwardyoon@apache.org
> >> <javascript:;>>
> >> >> wrote:
> >> >>> Once we change the iterable input messages to the vector, we can
> >> >>> change the legacy code like below:
> >> >>>
> >> >>> public void forward(FloatVector input) {
> >> >>>   float sum = input.dot(this.getWeightVector());
> >> >>>   this.feedforward(this.squashingFunction.apply(sum));
> >> >>> }
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Sat, Sep 3, 2016 at 11:10 PM, Baran Topal <
> >> barantopal@barantopal.com <javascript:;>>
> >> >>> wrote:
> >> >>>> Sure.
> >> >>>>
> >> >>>> In the attached, TestNeuron.txt,
> >> >>>>
> >> >>>> 1) I put // baran as a comment for the added functions.
> >> >>>>
> >> >>>> 2) The added functions and created objects have _ as suffix
> >> >>>>
> >> >>>> (e.g. backward_)
> >> >>>>
> >> >>>>
> >> >>>> A correction: above test execution time values were via
> >> >>>> System.nanoTime().
> >> >>>>
> >> >>>> Br.
> >> >>>>
> >> >>>> 2016-09-03 14:05 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org
> >> <javascript:;>>:
> >> >>>>> Interesting. Can you share your test code?
> >> >>>>>
> >> >>>>> On Sat, Sep 3, 2016 at 2:17 AM, Baran Topal <
> >> barantopal@barantopal.com <javascript:;>>
> >> >>>>> wrote:
> >> >>>>>> Hi Edward and team;
> >> >>>>>>
> >> >>>>>> I had a brief test by refactoring Iterable to Vector
and on
> >> >>>>>> TestNeuron.java, I can see some improved times. I didn't
check
> for
> >> >>>>>> other existing test methods but it seems the execution
times are
> >> >>>>>> improving for both forwarding and backwarding.
> >> >>>>>>
> >> >>>>>> These values are via System.currentTimeMillis().
> >> >>>>>>
> >> >>>>>> E.g.
> >> >>>>>>
> >> >>>>>>
> >> >>>>>> Execution time for the forward function is: 5722329
> >> >>>>>> Execution time for the backward function is: 31825
> >> >>>>>>
> >> >>>>>> Execution time for the refactored forward function
is: 72330
> >> >>>>>> Execution time for the refactored backward function
is: 4665
> >> >>>>>>
> >> >>>>>> Br.
> >> >>>>>>
> >> >>>>>> 2016-09-02 2:14 GMT+02:00 Yeonhee Lee <ssallys0130@gmail.com
> >> <javascript:;>>:
> >> >>>>>>> Hi Edward,
> >> >>>>>>>
> >> >>>>>>> If we don't have that kind of method in the neuron,
I guess it's
> >> >>>>>>> appropriate to put the method to the neuron.
> >> >>>>>>> That can be one of the distinct features of Horn.
> >> >>>>>>>
> >> >>>>>>> Regards,
> >> >>>>>>> Yeonhee
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> 2016-08-26 9:40 GMT+09:00 Edward J. Yoon <
> edward.yoon@samsung.com
> >> <javascript:;>>:
> >> >>>>>>>
> >> >>>>>>>> Hi forks,
> >> >>>>>>>>
> >> >>>>>>>> Our current neuron API is designed like:
> >> >>>>>>>> https://github.com/apache/incubator-horn/blob/master/
> >> >>>>>>>> README.md#programming-m
> >> >>>>>>>> odel
> >> >>>>>>>>
> >> >>>>>>>> In forward() method, each neuron receives the
pairs of the
> inputs
> >> x1,
> >> >>>>>>>> x2,
> >> >>>>>>>> ... xn from other neurons and weights w1, w2,
... wn like
> below:
> >> >>>>>>>>
> >> >>>>>>>>   public void forward(Iterable<M> messages)
throws IOException;
> >> >>>>>>>>
> >> >>>>>>>> Instead of this, I suggest that we use just
vector like below:
> >> >>>>>>>>
> >> >>>>>>>>   /**
> >> >>>>>>>>    * @param input vector from other neurons
> >> >>>>>>>>    * /
> >> >>>>>>>>   public void forward(Vector input) throws
IOException;
> >> >>>>>>>>
> >> >>>>>>>> And, the neuron provides a getWeightVector()
method that
> returns
> >> >>>>>>>> weight
> >> >>>>>>>> vector associated with itself. I think this
is more make sense
> >> than
> >> >>>>>>>> current
> >> >>>>>>>> version, and more easy to use GPU in the future.
> >> >>>>>>>>
> >> >>>>>>>> What do you think?
> >> >>>>>>>>
> >> >>>>>>>> Thanks.
> >> >>>>>>>>
> >> >>>>>>>> --
> >> >>>>>>>> Best Regards, Edward J. Yoon
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>> --
> >> >>>>> Best Regards, Edward J. Yoon
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Best Regards, Edward J. Yoon
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards, Edward J. Yoon
> >>
> >>
> >>
> >>
> >
> >
>



Mime
View raw message