commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gilles <gil...@harfang.homelinux.org>
Subject Re: [Math] Would like to know, if extensions are needed
Date Wed, 24 Dec 2014 01:21:41 GMT
Hi.

On Thu, 18 Dec 2014 23:13:00 +0100, Thorsten Kiefer wrote:
> Hello,
>
> Am 03.12.2014 um 12:53 schrieb Gilles:
>> Hello.
>>
>> On Wed, 03 Dec 2014 10:13:08 +0100, Thorsten Kiefer wrote:
>>> Hi,
>>> the port is now finished.
>>> The code may still contain bugs.
>>> Maybe alpha-state.
>>> But if you like it, you can add it.
>>
>> Thank you for offering code to enhance the scope of Commons Math.
>> Yet I must emphasize that one of the goals of the project is to
>> provide an "integrated" set of functionalities (a.o. consistent
>> style and API throughout the codebase, full documentation and test
>> coverage).
>> Another on-going task is to include contributors who will stay and
>> maintain the codebase.
>> We do not usually have the resources neither to integrate a new
>> functionality, nor to maintain it afterwards.
>>
>> You are very welcome to explain the purpose and state of your
>> contribution; then we can start a discussion on how to move toward
>> including the code in Commons Math.
>
> The purpose of my project is a learning algorithm, which uses
> function approximation and reinforcement learning.
> With reinforcement learning a so-called agent can learn
> optimal behaviour from only receiving rewards
> for actions done in perceived states.
>
> At the moment I am having problems to stabilize the algorithms.
> The topology of the approximation functions and the initial
> values for the parameters are essential for success.
>
> At the moment only the fourier basis is practical as
> approximation function.
>
>
> Naively one would create a lookup table of scores for state-action 
> pairs.
> As the impractical huge size of these lookup tables is the problem,
> the lookup table is being interpolated by an approximation function.
>
> The lookup table and respectively the approximation function
> map state-action pairs to scorings.
>
> The RL algorithm adujsts the scorings during simulation.
> And then it can decide, which action to take, by aiming for the
> highest score.
>
> But still all unstable.
>

The project certainly looks interesting.
However, IMHO, there is still a lot of work for your code to
to be considered for incorporating into Commons Math; as I
indicated above, it lacks documentation, references, unit
tests...

A possible step would be to use Commons Math whenever it
provides an alternative to yours (cf. "DiffableFunction" vs.
CM's "UnivariateDifferentiableFunction", "Cosine" vs CM's
"Cos", etc.).
[This would certainly help reviewers since the code would
contain already known bits, but more importantly, it will
avoid feature (and code) duplication.]
Then let us know where CM is lacking required functionalities,
and provide the missing features as independent contributions.
[They would be more easily grasped, discussed, and decided on.
And it will also made it easier to eventually review the bulk
of your project since several parts would have already been
included.]


Best,
Gilles

>>> [...]


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message