flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shuyi Chen <suez1...@gmail.com>
Subject Re: env.execute() ?
Date Wed, 30 May 2018 21:37:51 GMT
I think you might be looking for the functionality provided by the
clusterclient [1]. But I am not sure if I fully understand the meaning of
"do internally in sync with application". Maybe you can give a concrete use
case, so we can help better, if the ClusterClient is not what you want.

[1]
https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/client/program/ClusterClient.html

On Wed, May 30, 2018 at 3:18 AM, Esa Heikkinen <esa.heikkinen@student.tut.fi
> wrote:

> Hi
>
>
>
> Ok. Thanks for the clarification. But the controlling of savepoints is
> only possible by command line (or a script) ? Or is it possible to do
> internally in sync with application ?
>
>
>
> Esa
>
>
>
> *From:* Shuyi Chen <suez1224@gmail.com>
> *Sent:* Wednesday, May 30, 2018 8:18 AM
> *To:* Esa Heikkinen <esa.heikkinen@student.tut.fi>
> *Cc:* Fabian Hueske <fhueske@gmail.com>; user@flink.apache.org
> *Subject:* Re: env.execute() ?
>
>
>
> Hi Esa,
>
>
>
> I think having more than one env.execute() is anti-pattern in Flink.
>
>
>
> env.execute() behaves differently depending on the env. For local, it will
> generate the flink job graph, and start a local mini cluster in background
> to run the job graph directly.
> For remote case, it will generate the flink job graph and submit it to a
> remote cluster, e.g. running on YARN/Mesos, the local process might stay
> attached or detach to the job on the remote cluster given options. So it's
> not a simple "unstoppable forever loop", and I dont think the "stop
> env.execute() and then do something and after that restart it" will work in
> general.
>
>
>
> But I think you can take a look at savepoints [1] and checkpoints [2] in
> Flink. With savepoints, you can stop the running job, and do something
> else, and restart from the savepoints to resume the processing.
>
>
>
>
>
> [1]  https://ci.apache.org/projects/flink/flink-docs-
> release-1.5/ops/state/savepoints.html
>
> [2] https://ci.apache.org/projects/flink/flink-docs-release-1.5/ops/state/
> checkpoints.html
>
>
>
> Thanks
>
> Shuyi
>
>
>
> On Tue, May 29, 2018 at 3:56 AM, Esa Heikkinen <
> esa.heikkinen@student.tut.fi> wrote:
>
> Hi
>
>
>
> Are there only one env.execute() in application ?
>
>
>
> Is it unstoppable forever loop ?
>
>
>
> Or can I stop env.execute() and then do something and after that restart
> it ?
>
>
>
> Best, Esa
>
>
>
> *From:* Fabian Hueske <fhueske@gmail.com>
> *Sent:* Tuesday, May 29, 2018 1:35 PM
> *To:* Esa Heikkinen <esa.heikkinen@student.tut.fi>
> *Cc:* user@flink.apache.org
> *Subject:* Re: env.execute() ?
>
>
>
> Hi,
>
>
>
> It is mandatory for all DataStream programs and most DataSet programs.
>
>
>
> Exceptions are ExecutionEnvironment.print() and
> ExecutionEnvironment.collect().
>
> Both methods are defined on the DataSet ExecutionEnvironment and call
> execute() internally.
>
>
>
> Best, Fabian
>
>
>
> 2018-05-29 12:31 GMT+02:00 Esa Heikkinen <esa.heikkinen@student.tut.fi>:
>
> Hi
>
>
>
> Is it env.execute() mandatory at the end of application ? It is possible
> to run the application without it ?
>
>
>
> I found some examples where it is missing.
>
>
>
> Best, Esa
>
>
>
>
>
>
>
> --
>
> "So you have to trust that the dots will somehow connect in your future."
>



-- 
"So you have to trust that the dots will somehow connect in your future."

Mime
View raw message