flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Convergence Criterion in IterativeDataSet
Date Fri, 04 Sep 2015 12:54:32 GMT
I think you can do this with the current interface. The convergence
criterion object stays around, so you should be able to simply store the
current aggregator value in a field (when the check is invoked). Any round
but the first could compare against that field.

On Fri, Sep 4, 2015 at 2:25 PM, Andres R. Masegosa <andres@cs.aau.dk> wrote:

> Hi,
> I trying to implement some machine learning algorithms that involve
> several iterations until convergence (to a fixed point).
> My idea is to use a IterativeDataSet with an Aggregator which produces
> the result (i.e. a set of parameters defining the model).
> From the interface "ConvergenceCriterion", I can understand that the
> convergence criterion only depends on the result of the aggregator in
> the current iteration (as happens with the DoubleZeroConvergence class).
> However, it is more usual to test convergence by comparing the result of
> the aggregator in the current iteration with the result of the
> aggregator in the previous iteration (one usually stops when both
> results are similar enough and we have converged to a fixed point).
> I guess this functionality is not included yet. And this is because the
> convergence criteria of flink implementations of K-Means and Linear
> Regression is to stop after a fixed number of iterations.
> Am I wrong?
> Regards
> Andres

View raw message