flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Szabó Péter <nemderogator...@gmail.com>
Subject Re: [DISCUSS] Iterative streaming example
Date Mon, 23 Feb 2015 13:29:51 GMT
Nice. Thank you guys!

@Paris
Are there any Flink implementations of this model? The GitHub doc is quite
general.

Peter

2015-02-23 14:05 GMT+01:00 Paris Carbone <parisc@kth.se>:

> Hello Peter,
>
> Streaming machine learning algorithms make use of iterations quite widely.
> One simple example is implementing distributed stream learners. There, in
> many cases you need some central model aggregator, distributed estimators
> to offload the central node and of course feedback loops to merge
> everything back to the main aggregator periodically. One such example in
> the Vertical Hoeffding Tree Classifier (VFDT) [1] that is implemented in
> Samoa.
>
> Iterative streams are also useful for optimisation techniques as in batch
> processing (eg. trying different parameters to estimate a variable, getting
> back the accuracy from an evaluator and repeating until a condition is
> achieved).
>
> I hope this helps to get a general idea of where iterations can be used.
>
> [1] https://github.com/yahoo/samoa/wiki/Vertical-Hoeffding-Tree-Classifier
>
>
> On 23 Feb 2015, at 12:13, Stephan Ewen <sewen@apache.org<mailto:
> sewen@apache.org>> wrote:
>
> I think that the Samoa people have quite a few nice examples along the
> lines of model training with feedback.
>
> @Paris: What would be the simplest example?
>
> On Mon, Feb 23, 2015 at 11:27 AM, Szabó Péter <nemderogatorius@gmail.com
> <mailto:nemderogatorius@gmail.com>>
> wrote:
>
> Does everyone know of a good, simple and realistic streaming iteration
> example? The current example tests a random generator, but it should be
> replaced by something deterministic in order to be testable.
>
> Peter
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message