commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luc Maisonobe <>
Subject Re: [nabla] INVOKEVIRTUAL not handled yet
Date Sun, 16 Oct 2011 18:07:20 GMT
Le 16/10/2011 18:10, Phil Steitz a écrit :
> On 10/16/11 8:01 AM, Luc Maisonobe wrote:
>> Le 16/10/2011 09:24, Phil Steitz a écrit :
>>> On 10/9/11 7:46 AM, Phil Steitz wrote:
>>>> On 10/9/11 5:39 AM, Luc Maisonobe wrote:
>>>>> Hi Phil,
>>>>> Le 08/10/2011 23:42, Phil Steitz a écrit :
>>>>>> On 10/8/11 2:24 PM, Luc Maisonobe wrote:
>>>>>>> Phil Steitz<>    a écrit :
>>>>>>>> I am getting RTE with message above when I try to run the
>>>>>>>> example
>>>>>>>> under "updating the base and differentiated objects" in the
>>>>>>>> docs.
>>>>> Digging into the code, here are the bytecode operations that are
>>>>> not supported yet:
>>>>>      DALOAD, DASTORE:
>>>>>         element access in double arrays
>>>>>         field access (instance fields and class fields)
>>>>>         method calls
>>>>>         array creation
>>>>>>>> Is this example supposed to work with the code in trunk?
>>>>>>>> I am
>>>>>>> I'll look at this tomorrow, but I think for now you need to have
>>>>>>> a standalone function, it cannot be split
>>>>>>> as a main function calling subfunctions. The only allowed calls
>>>>>>> are the static methods from Math/StrictMath.
>>>>>>> I did not add our own FastMath, but it is trivial to do.
>>>>>>> Another limitation is that your function cannot store
>>>>>>> intermediate results as clas attributes yet.
>>>>>> Thanks, Luc!  What I was trying to illustrate was partial
>>>>>> derivatives, which IIUC you need something like that example
>>>>>> to do.
>>>>>> The following almost works:
>>>>>>       public void testPartialDerivatives() throws Exception {
>>>>>>            PartialFunction function = new PartialFunction(1);
>>>>>>            final UnivariateDerivative derivative = new
>>>>>> ForwardModeAlgorithmicDifferentiator().differentiate(function);
>>>>>>            DifferentialPair t = DifferentialPair.newVariable(1);
>>>>>>            Assert.assertEquals(3,
>>>>>> derivative.f(t).getFirstDerivative(), 0);
>>>>>>            Assert.assertEquals(2, derivative.f(t).getValue(), 0);
>>>>>>            function.setX(2);
>>>>>>            Assert.assertEquals(4,
>>>>>> derivative.f(t).getFirstDerivative(), 0);
>>>>>>            Assert.assertEquals(3, derivative.f(t).getValue(), 0);
>>>>>>        }
>>>>>> with
>>>>>> public class PartialFunction implements
>>>>>> UnivariateDifferentiable {
>>>>>>        private double x;
>>>>>>        public PartialFunction(double x) {
>>>>>>            this.x = x;
>>>>>>        }
>>>>>>        public void setX(double x) {
>>>>>>            this.x = x;
>>>>>>        }
>>>>>>        public double getX() {
>>>>>>            return x;
>>>>>>        }
>>>>>>        public double f(double y) {
>>>>>>            return x * y + y * y;
>>>>>>        }
>>>>>> }
>>>>>> But I end up with java.lang.VerifyError: (class:
>>>>>> ExampleTest$1PartialFunction$NablaForwardModeUnivariateDerivative,
>>>>>> method: f signature:
>>>>>> (Lorg/apache/commons/nabla/core/DifferentialPair;)Lorg/apache/commons/nabla/core/DifferentialPair;)
>>>>>> Incompatible type for getting or setting field
>>>>>>        at java.lang.Class.getDeclaredConstructors0(Native Method)
>>>>>>        at
>>>>>> java.lang.Class.privateGetDeclaredConstructors(
>>>>>>        at java.lang.Class.getDeclaredConstructors(
>>>>>>        at
>>>>>> org.apache.commons.nabla.algorithmic.forward.ForwardModeAlgorithmicDifferentiator.differentiate(
>>>>>>        at ExampleTest.testPartialDerivatives(
>>>>> This error seems to be due to the lack of support for the GETFIELD
>>>>> instruction. As x is an instance field, the f method reads this
>>>>> field before multiplying the result.
>>> Right.  As an exercise to help me understand the internals of the
>>> code, I have been trying to figure out how to add this support.
>>> Working backwards from the generated bytecode, the GETFIELD and the
>>> ALOAD 0 before it seem to get copied unchanged:
>>>       ALOAD 0
>>>       GETFIELD PartialFunction.x : D
>>> Working backwards to where it should get changed,
>>> MethodDifferentiator#getReplacement throws RuntimeException when it
>>> sees a GETFIELD; but in my example it does not throw.  This means
>>> the instruction is not making it into the changes set.  The question
>>> then is a) how to modify MethodDifferentiator#identifyChanges to
>>> identify the need to change the instruction
>> Changes are deduced from a data flow analysis. We start at method
>> entry, knowing that the original method signature was
>>    public double f(double x);
>> and that the transformed method signature is
>>    public DifferentialPair f(DifferentialPair x);
>> So on entry, the first (and only) parameter is changed from double
>> to DifferentialPair.
>>  From this starting point, we propagate types, using
>> differentiation rules, i.e. when we see that an instruction like
>> "add" has a DifferentialPair at least in one of its arguments,
>> then its result must be a DifferentialPair. At the end of the data
>> flow analysis, we have identified all instructions that either
>> consume or produce DifferentialPair instances, these instructions
>> must be changed as in the original method they did consume or
>> produce double number instead.
>>> and b) how to transform
>>> it (and other instructions that depend on it).
>> Transform may depend on context. For example, if the field is only
>> read in the method and never set (which is the case in this
>> example, as the method computes x * y + y * y where x is the field
>> and y is the method parameter), then this field should remain a
>> simple double and the instruction using it should be changed from
>> multiplication of two doubles (x * y) to a multiplication of a
>> double and a DifferentialPair. If the field were also set (for
>> example using setX(y * y)), then it would be the result of an
>> instruction producing a DifferentialPair and an additional field
>> should be set up in the generated class. We could either choose to
>> set a complete "private DifferentialPair xNabla" containing both
>> the value and the derivative, with some code to make sure the x
>> field from the enclosing class is kept in sync, or we could simply
>> use a "private double xNabla" for the derivative and still use the
>> x from the enclosing class for the value, hence avoiding sync code.
> If possible, I would think it best to avoid the sync code.  This
> would also eliminate the need to check for changes made by other
> clients to the primitive field.

I agree.

>>> A reference to the
>>> enclosing class also has to be made available.  Is that already
>>> there somewhere?
>> Yes, there is a "primitive" field in the generated class that
>> points to the original instance. This field is the one returned by
>> the automatically generated "getPrimitive" method.
> Duh...Should have seen that.  Thanks.  Quick bytecode question.
> Here is the generated init for the derivative class, showing where
> the primitive field is set:
>    public<init>(LPartialFunction;)V
>      ALOAD 0
>      INVOKESPECIAL java/lang/Object.<init>  ()V
>      ALOAD 0
>      ALOAD 1
> PartialFunction$NablaForwardModeUnivariateDerivative.primitive :
> LPartialFunction;
> I am a novice when it comes to bytecode analysis.  From the above,
> it looks like the enclosing class is available using ALOAD 1.  Is
> this always the case?  If that is true, then might the incorrect
> field access bytecode above be fixed just by changing 0 to 1 in the

No, it's not always the case. On method invocation, the local variables 
correspond to the arguments list. For an instance method, local variable 
0 is "this" and local variable 1 is the first argument, here it is the 
PartialFunction instance provided as an argument to the constructor. 
This constructor is invoked automatically by Nabla and it is indeed the 
primitive instance.

When the f method is invoked, we still have this as the variable 0, and 
x as the argument, hence as variable 1. As long as we do not override 
variable 0, we can retrieve this from there. So to get the x field from 
the primitive, I guess we should do "ALOAD 0" to get this, then 
"GETFIELD primitive" to get the enclosing instance, and then "GETFIELD 
x" to retrive the x field value itself.

>> I think there is a problem in the data flow analysis, in the
>> TrackingValue class. It's "merge" is never populated with
>> anything. I will fix this.
>> I will also try to improve the debugging prints I have added a few
>> days ago (creating a new class for it). It should also display the
>> status of local variables and stack, as these are the main drivers
>> for the data flow analysis and it is probably where the error lies
>> here.
> Thanks, Luc. I am starting to understand how things work and I have
> to say there are some really nice ideas in here.  Regarding the data
> flow analysis, at first I thought there was no way that the GETFIELD
> could get picked up because the data flow analysis only looks at t;
> but I can see now that assuming the field is actually used, the
> instruction should get picked up.  I will look at this some more.

You are right, the data flow analysis is not sufficient here. In fact, 
we should not only track what comes from the method parameters (i.e. the 
local variables), but also make sure that a GETFIELD that refers to the 
original instance should be either split in two GETFIELD if the variable 
is extended to include a deriviative (when there is another PUTFIELD 
elsewhere), or that event if the field is only read, that we try to 
recover it from the primitive and not from the instance itself.

> Regarding debugging, what I have found useful is dumping the
> original and derived bytecode.  I have started dumping the changes
> set too to understand the data flow analysis.
> One final sort of philosophical question.  As you point out in the
> docs, one could actually build up DifferentialPairs using the
> primitives defined on them.  These are not used internally and you
> remark that using this method to construct things might be error
> prone.  A hybrid model where (lets call them non-degenerate)
> DifferentialPairs are provided as arguments to derived functions is
> also possible.  The derivatives will be correctly evaluated and
> applied according to the chain rule.  Is this the kind of thing you
> had in mind and, if so, what kinds of applications would use this?

I'm not sure. What I had in mind was that a user might rely on Nabla to 
build some parts of a function and use DifferentialPairs to assemble 
these parts by himslef. For example, if a user has two classes F ang G, 
he could differentiate both to get automatically two instances, and 
directly create some combined function (says f' + g') without being 
forced to build first f+g and differentiate it. However, I'm not sure 
this is useful, perhaps having a basic DifferentialPair that is only a 
container and forcing the user to first build a complete implementation 
of the primitive function and then to differentiate it would be simpler.


> Phil
>> Luc
>>> Phil
>>>>> I have added a debug display message (to be removed later on) that
>>>>> should print the generated bytecode to standard error when a
>>>>> VerifyError exception occurs. It' clearly not targeted towards end
>>>>> users, but it could help during development.
>>>> Thanks, Luc!
>>>> Phil
>>>>> Luc
>>>>>>> You can look at the junit tests for what is supported.  Simple
>>>>>>> expressions, calls to traditional functions like sin, cos,
>>>>>>> exp ...,
>>>>>>> Simple loops and conditionals, local automatic variables should
>>>>>>> all work (I hope ...)
>>>>>> Yep, I have gotten all of this to work.  Even "knows" the chain
>>>>>> rule :)
>>>>>> Phil
>>>>>>>> assuming
>>>>>>>> s/ForwardAlgorithmicDifferentiator/ForwardModeAlgorithmicDifferentiator
>>>>>>>> throughout.  Correct?
>>>>>>> Yes, the name was changed because a distant goal will be to also
>>>>>>> support reverse mode, which is especially
>>>>>>> useful when computing gradients (i.e. when one scalar function
>>>>>>> depends on many inputs and we want all partial
>>>>>>> derivatives).
>>>>>>> Luc
>>>>>>>> Phil
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail:
>>>>>>>> For additional commands, e-mail:
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail:
>>>>>>> For additional commands, e-mail:
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail:
>>>>>> For additional commands, e-mail:
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail:
>>>>> For additional commands, e-mail:
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>>> For additional commands, e-mail:
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message