flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Asterios Katsifodimos <asterios.katsifodi...@tu-berlin.de>
Subject Re: Lazy unrolling of iterative algorithms
Date Thu, 30 Oct 2014 13:36:49 GMT
We woudl like to have asap actually.

Our deadline is on Thursday. However, I think I could potentially work on
the JobGraph.

What we would like to see is the effect of re-deploying a job (a la Spark)
on every iteration and measure it in large deployments (i.e. the overhead
of not persisting tasks on machines).

Do you think we could "simulate" this in an easy way?


On Thursday, October 30, 2014, Stephan Ewen <sewen@apache.org> wrote:

>  Hey!
>  The Java API does not yet support it, the JobGraph supports it, though.
>  In the runtime, it will still execute everything from the start if you
> "attach" another rolled out version of the loop."
>  We should have that working in a few weeks.
>  What is your schedule? How fast do you need it?
>  Stephan
> On Thu, Oct 30, 2014 at 11:43 AM, Asterios Katsifodimos <
> asterios.katsifodimos@tu-berlin.de> wrote:
>> Hello,
>>  I would like to use the feature of lazy unrolling in order to run
>> KMeans.
>>  Would this be possible in he current API? Or should I use a lower level
>> API (what we used to call Nephele?)
>>  Thanks,
>> Asterios

View raw message