Hi users,
Sorry for the long problem description.
I implemented a radial basis function network for nonlinear
regression with adaptive centers and adaptive basis shapes (diagonal
covariance matrix) using the LevenbergMarquardt solver
(org.apache.commons.math3.optim.nonlinear.vector.jacobian.LevenbergMarquardtOptimizer)
and the ModelFunction's
DerivativeStructure[] value(DerivativeStructure[] x)
function using the DerivativeStructure API so that the derivatives are
computed analytically.
For a reasonable sized network with 200 radial bases, the number of
parameters is
(200 /* # bases */ +1 /* bias */ +((dim /* center of 1 basis */ + dim
/* shape parameters of 1 basis */)*200))
where "dim" is the dimension of the input vectors. This results in a
few hundred free parameters. For small amounts of data, everything
works fine. But in problems with highdimensional input, I sometimes
use tens of thousands (or even hundreds of thousands) of training
samples. Unfortunately, with this much training data, I receive either
a Java Heap Error or a Garbage Collection Error (in the middle of
differentiation).
The main problem seems to be that the optimizer expects the
ModelFunction to return a vector evaluating all of the training
samples to compare with the Target instance passed in as
OptimizationData. For regular evaluation this isn't to much of a
problem, but the memory used by the DerivativeStructure instances
(spread out over a few hundred parameters times 10,000 evaluations) is
massive.
Is there any way to get the solver to evaluate the residuals/gradient
incrementally?
Thank you for the advice in advance.
Timothy A. Mann

To unsubscribe, email: userunsubscribe@commons.apache.org
For additional commands, email: userhelp@commons.apache.org
