flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: Towards Flink 0.10
Date Tue, 06 Oct 2015 10:29:09 GMT
@Stephan: There is an issue for implementing a detached job submission:
https://issues.apache.org/jira/browse/FLINK-2797. That has the advantage
that the execution mode and control flow is not hard-coded in the user job.
I think it makes more sense to integrate methods like
(cancel/getAccumulators/scale) in the Client class and then also make them
accessible via the command-line client. IMHO we could think about a special
executeWithControl() method for these things but I would not break the
existing execute().

On Tue, Oct 6, 2015 at 12:18 PM, Alexey Sapozhnikov <alexey@scalabill.it>
wrote:

> Hello everyone.
>
> Stephan, will 0.10 include the cache issues for createRemoteEnvironment?
> That not every addressing of environment will retransmit all the jars?
>
> On Tue, Oct 6, 2015 at 1:16 PM, Stephan Ewen <sewen@apache.org> wrote:
>
> > How about making a quick effort for this:
> > https://issues.apache.org/jira/browse/FLINK-2313
> >
> > It introduces an execution control handle (returned by
> > StreamExecutionEnvironment.execute()) for operations like cancel(),
> > getAccumulators(), scaleIn/Out(), ...
> > It would be big time API breaking, so would be good to have it before the
> > release.
> >
> > As a first version we could return a control handle that has only a
> single
> > method "waitForCompletion()" to emulate the current behavior.
> > Or we postpone it and later add a method "executeWithControl()" or so,
> that
> > returns the control handle.
> >
> >
> > On Mon, Oct 5, 2015 at 6:34 PM, Vasiliki Kalavri <
> > vasilikikalavri@gmail.com>
> > wrote:
> >
> > > Yes, FLINK-2785 that's right!
> > > Alright, thanks a lot!
> > >
> > > On 5 October 2015 at 18:31, Fabian Hueske <fhueske@gmail.com> wrote:
> > >
> > > > Hi Vasia,
> > > >
> > > > I guess you are referring to FLINK-2785. Should be fine, as there is
> a
> > PR
> > > > already.
> > > > I'll add it to the list.
> > > >
> > > > Would be nice if you could take care of FLINK-2786 (remove Spargel).
> > > >
> > > > Cheers, Fabian
> > > >
> > > > 2015-10-05 18:25 GMT+02:00 Vasiliki Kalavri <
> vasilikikalavri@gmail.com
> > >:
> > > >
> > > > > Thank you Max for putting the list together and to whomever added
> > > > > FLINK-2561 to the list :)
> > > > > I would also add FLINK-2561 (pending PR #1205). It's a sub-task of
> > > > > FLINK-2561, so maybe it's covered as is.
> > > > >
> > > > > If we go for Gelly graduation, I can take care of FLINK-2786
> "Remove
> > > > > Spargel from source code and update documentation in favor of
> Gelly",
> > > but
> > > > > maybe it makes sense to wait for the restructuring, since we're
> > getting
> > > > rid
> > > > > of staging altogether?
> > > > >
> > > > > -V.
> > > > >
> > > > > On 5 October 2015 at 17:51, Fabian Hueske <fhueske@gmail.com>
> wrote:
> > > > >
> > > > > > Thanks Max.
> > > > > > I extended the list of issues to fix for the release.
> > > > > >
> > > > > > 2015-10-05 17:10 GMT+02:00 Maximilian Michels <mxm@apache.org>:
> > > > > >
> > > > > > > Thanks Greg, we have added that to the list of API breaking
> > > changes.
> > > > > > >
> > > > > > > On Mon, Oct 5, 2015 at 4:36 PM, Greg Hogan <code@greghogan.com
> >
> > > > wrote:
> > > > > > >
> > > > > > > > Max,
> > > > > > > >
> > > > > > > > Stephan noted that FLINK-2723 is an API breaking change.
The
> > > > > > > CopyableValue
> > > > > > > > interface has a new method "T copy()". Commit
> > > > > > > > e727355e42bd0ad7d403aee703aaf33a68a839d2
> > > > > > > >
> > > > > > > > Greg
> > > > > > > >
> > > > > > > > On Mon, Oct 5, 2015 at 10:20 AM, Maximilian Michels
<
> > > > mxm@apache.org>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Flinksters,
> > > > > > > > >
> > > > > > > > > After a lot of development effort in the past
months, it is
> > > about
> > > > > > time
> > > > > > > > > to move towards the next major release. We decided
to move
> > > > towards
> > > > > > > > > 0.10 instead of a milestone release. This release
will
> > probably
> > > > be
> > > > > > the
> > > > > > > > > last release before 1.0.
> > > > > > > > >
> > > > > > > > > For 0.10 we most noticeably have the new Streaming
API
> which
> > > > comes
> > > > > > > > > with an improved runtime including exactly-once
sources and
> > > > sinks.
> > > > > > > > > Additionally, we have a new web interface with
support for
> > > > > > > > > live-monitoring. Not to mention the countless
fixes and
> > > > > improvements.
> > > > > > > > >
> > > > > > > > > I've been ransacking the JIRA issues to find
out what
> issues
> > we
> > > > > have
> > > > > > > > > to fix before we can release. I've put these
issues on the
> > 0.10
> > > > > wiki
> > > > > > > > > page:
> > > > > https://cwiki.apache.org/confluence/display/FLINK/0.10+Release
> > > > > > > > >
> > > > > > > > > It would be great to fix all of these issues.
However, I
> > think
> > > we
> > > > > > need
> > > > > > > > > to be pragmatic and pick the most pressing ones.
Let's do
> > that
> > > on
> > > > > the
> > > > > > > > > wiki page on the "To Be Fixed" section.
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > > Max
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
>
>
> --
>
> *Regards*
>
> *Alexey Sapozhnikov*
> CTO& Co-Founder
> Scalabillit Inc
> Aba Even 10-C, Herzelia, Israel
> M : +972-52-2363823
> E : alexey@scalabill.it
> W : http://www.scalabill.it
> YT - https://youtu.be/9Rj309PTOFA
> Map:http://mapta.gs/Scalabillit
> Revolutionizing Proof-of-Concept
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message