horn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edward.y...@samsung.com>
Subject RE: Use vector instead of Iterable in neuron API
Date Tue, 06 Sep 2016 22:41:21 GMT
> directly into FloatVector and get rid of Synapse totally?

Yes, I think that's more clear.

Internally, each task processes neurons' computation on assigned data split. 
The LayeredNeuralNetwork.java contains everything, you can find the 
"n.forward(msg);" and "n.backward(msg);" at there. Before calling the 
forward() or backward() method, we can set the weight vector associated with 
neuron using setWeightVector() method and set the argument value for the 
forward() method like below:

n.setWeightVector(weight vector e.g., weightMatrix.get(neuronID));
n.forward(outputs from previous layer as a FloatVector);

Then, user-side program can be done like below:

forward(FloatVector input) {
  this.getWeightVector(); // returns weight vector associated with itself.
  float vectorSum;
  for(float element : input) {
    vectorSum += element;
  }
}

If you have any trouble, please don't hesitate ask here.

--
Best Regards, Edward J. Yoon


-----Original Message-----
From: Baran Topal [mailto:barantopal@barantopal.com]
Sent: Tuesday, September 06, 2016 11:42 PM
To: dev@horn.incubator.apache.org
Subject: Re: Use vector instead of Iterable in neuron API

Hi team and Edward;

I have been checking this and got stuck how to convert the list
structure to DenseFloatVector. Can you help on this?

Let me explain:

I saw that the concrete FloatVector is actually some sort of array
structure which is not really compatible with Synapse and
FloatWritables. Is the aim to convert Synapse construction logic
directly into FloatVector and get rid of Synapse totally?

//

In the original code, we are passing a list with a structure of having
Synapse and FloatWritables. I can see that a synapse can be
constructed with a neuron id and 2 float writables.

What I tried;

1) I added the following function in Neuron.java

  public DenseFloatVector getWeightVector() {
    DenseFloatVector dfv = new DenseFloatVector(getWeights());
    return dfv;
  }

2) I added the following function in NeuronInterface.java

public void forward2(FloatVector messages) throws IOException;

3) I added the following function in TestNeuron.java

    public void forward2(FloatVector messages) throws IOException {

      long startTime = System.nanoTime();

      float sum = messages.dot(this.getWeightVector());
      this.feedforward(this.squashingFunction.apply(sum));

      long endTime = System.nanoTime();

      System.out.println("Execution time for the forward2 function is:
" + ((endTime - startTime)));

    }
4) I tried to refactor testProp() in TestNeuron.java with several ways
but failed unfortunately with runtime error since weight is not having
a value.

  MyNeuron n_ = new MyNeuron();

    FloatVector ds = new DenseFloatVector();

    Iterator <Synapse<FloatWritable, FloatWritable>> li = x_.iterator();
    // Iterator<FloatVector.FloatVectorElement> ie = ds.iterate();

    while(li.hasNext()) {
      // ds.add(li.next());
      // ie.
      Synapse<FloatWritable, FloatWritable> ee = li.next();

      ds.set(ee.getSenderID(), ee.getMessage());

    }

    float[] ff = new float[2];
    ff[0] = 1.0f;
    ff[1] = 0.5f;

    float[] ffa = new float[2];
    ffa[0] = 1.0f;
    ffa[1] = 0.4f;

    DenseFloatVector dss = new DenseFloatVector(ff);
    DenseFloatVector dssa = new DenseFloatVector(ffa);

    dss.add(dssa);


    FloatWritable a = new FloatWritable(1.0f);
    FloatWritable b = new FloatWritable(0.5f);

    Synapse s = new Synapse(0, a, b);



  //  dss.set(1, 0.5f);
    //dss.set(1, 0.4f);

   // DenseFloatVector ds = new DenseFloatVector(ff);

    n_.forward2(dss); //forward2



2016-09-05 0:55 GMT+02:00 Baran Topal <barantopal@barantopal.com>:
> Hi;
>
> Thanks I am on it.
>
> Br.
>
> 2016-09-04 4:16 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org>:
>> P.S., so, if you want to test more, please see FloatVector and
>> DenseFloatVector.
>>
>> On Sun, Sep 4, 2016 at 11:13 AM, Edward J. Yoon <edwardyoon@apache.org> 
>> wrote:
>>> Once we change the iterable input messages to the vector, we can
>>> change the legacy code like below:
>>>
>>> public void forward(FloatVector input) {
>>>   float sum = input.dot(this.getWeightVector());
>>>   this.feedforward(this.squashingFunction.apply(sum));
>>> }
>>>
>>>
>>>
>>>
>>> On Sat, Sep 3, 2016 at 11:10 PM, Baran Topal <barantopal@barantopal.com>

>>> wrote:
>>>> Sure.
>>>>
>>>> In the attached, TestNeuron.txt,
>>>>
>>>> 1) I put // baran as a comment for the added functions.
>>>>
>>>> 2) The added functions and created objects have _ as suffix
>>>>
>>>> (e.g. backward_)
>>>>
>>>>
>>>> A correction: above test execution time values were via 
>>>> System.nanoTime().
>>>>
>>>> Br.
>>>>
>>>> 2016-09-03 14:05 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org>:
>>>>> Interesting. Can you share your test code?
>>>>>
>>>>> On Sat, Sep 3, 2016 at 2:17 AM, Baran Topal <barantopal@barantopal.com>

>>>>> wrote:
>>>>>> Hi Edward and team;
>>>>>>
>>>>>> I had a brief test by refactoring Iterable to Vector and on
>>>>>> TestNeuron.java, I can see some improved times. I didn't check for
>>>>>> other existing test methods but it seems the execution times are
>>>>>> improving for both forwarding and backwarding.
>>>>>>
>>>>>> These values are via System.currentTimeMillis().
>>>>>>
>>>>>> E.g.
>>>>>>
>>>>>>
>>>>>> Execution time for the forward function is: 5722329
>>>>>> Execution time for the backward function is: 31825
>>>>>>
>>>>>> Execution time for the refactored forward function is: 72330
>>>>>> Execution time for the refactored backward function is: 4665
>>>>>>
>>>>>> Br.
>>>>>>
>>>>>> 2016-09-02 2:14 GMT+02:00 Yeonhee Lee <ssallys0130@gmail.com>:
>>>>>>> Hi Edward,
>>>>>>>
>>>>>>> If we don't have that kind of method in the neuron, I guess it's
>>>>>>> appropriate to put the method to the neuron.
>>>>>>> That can be one of the distinct features of Horn.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Yeonhee
>>>>>>>
>>>>>>>
>>>>>>> 2016-08-26 9:40 GMT+09:00 Edward J. Yoon <edward.yoon@samsung.com>:
>>>>>>>
>>>>>>>> Hi forks,
>>>>>>>>
>>>>>>>> Our current neuron API is designed like:
>>>>>>>> https://github.com/apache/incubator-horn/blob/master/
>>>>>>>> README.md#programming-m
>>>>>>>> odel
>>>>>>>>
>>>>>>>> In forward() method, each neuron receives the pairs of the
inputs x1, 
>>>>>>>> x2,
>>>>>>>> ... xn from other neurons and weights w1, w2, ... wn like
below:
>>>>>>>>
>>>>>>>>   public void forward(Iterable<M> messages) throws
IOException;
>>>>>>>>
>>>>>>>> Instead of this, I suggest that we use just vector like below:
>>>>>>>>
>>>>>>>>   /**
>>>>>>>>    * @param input vector from other neurons
>>>>>>>>    * /
>>>>>>>>   public void forward(Vector input) throws IOException;
>>>>>>>>
>>>>>>>> And, the neuron provides a getWeightVector() method that
returns 
>>>>>>>> weight
>>>>>>>> vector associated with itself. I think this is more make
sense than 
>>>>>>>> current
>>>>>>>> version, and more easy to use GPU in the future.
>>>>>>>>
>>>>>>>> What do you think?
>>>>>>>>
>>>>>>>> Thanks.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best Regards, Edward J. Yoon
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards, Edward J. Yoon
>>>
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon




Mime
View raw message