flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paris Carbone <par...@kth.se>
Subject Re: [DISCUSS] Iterative streaming example
Date Mon, 23 Feb 2015 13:05:22 GMT
Hello Peter,

Streaming machine learning algorithms make use of iterations quite widely. One simple example
is implementing distributed stream learners. There, in many cases you need some central model
aggregator, distributed estimators to offload the central node and of course feedback loops
to merge everything back to the main aggregator periodically. One such example in the Vertical
Hoeffding Tree Classifier (VFDT) [1] that is implemented in Samoa.

Iterative streams are also useful for optimisation techniques as in batch processing (eg.
trying different parameters to estimate a variable, getting back the accuracy from an evaluator
and repeating until a condition is achieved).

I hope this helps to get a general idea of where iterations can be used.

[1] https://github.com/yahoo/samoa/wiki/Vertical-Hoeffding-Tree-Classifier


On 23 Feb 2015, at 12:13, Stephan Ewen <sewen@apache.org<mailto:sewen@apache.org>>
wrote:

I think that the Samoa people have quite a few nice examples along the
lines of model training with feedback.

@Paris: What would be the simplest example?

On Mon, Feb 23, 2015 at 11:27 AM, Szabó Péter <nemderogatorius@gmail.com<mailto:nemderogatorius@gmail.com>>
wrote:

Does everyone know of a good, simple and realistic streaming iteration
example? The current example tests a random generator, but it should be
replaced by something deterministic in order to be testable.

Peter


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message