flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Connecting workflows in batch
Date Fri, 03 Mar 2017 15:36:41 GMT
Yes, right now that call never returns for a long-running streaming job.
We will (in the future) provide a way for that call to return so that
the result can be used for checking aggregators and other things.

On Thu, Mar 2, 2017, at 19:14, Mohit Anchlia wrote:

> Does it mean that for streaming jobs it never returns?


> On Thu, Mar 2, 2017 at 6:21 AM, Till Rohrmann
> <trohrmann@apache.org> wrote:
>> Hi Mohit,

>> StreamExecutionEnvironment.execute() will only return giving you the
>> JobExecutionResult after the job has reached a final stage. If that
>> works for you to schedule the second job, then it should be ok to
>> combine both jobs in one program and execute the second job after the
>> first one has completed.
>> Cheers,

>> Till



>> On Thu, Mar 2, 2017 at 2:33 AM, Mohit Anchlia
>> <mohitanchlia@gmail.com> wrote:
>>> It looks like JobExecutionResult can be used here by using the
>>> accumulators?

>>> On Wed, Mar 1, 2017 at 8:37 AM, Aljoscha Krettek
>>> <aljoscha@apache.org> wrote:
>>>> I think right now the best option is the JobManager REST interface:
>>>> https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/rest_api.html

>>>> You would have to know the ID of your job and then you can poll the
>>>> status of your running jobs.

>>>> On Mon, 27 Feb 2017 at 18:15 Mohit Anchlia <mohitanchlia@gmail.com>
>>>> wrote:
>>>>> What's the best way to track the progress of the job?


>>>>> On Mon, Feb 27, 2017 at 7:56 AM, Aljoscha Krettek
>>>>> <aljoscha@apache.org> wrote:
>>>>>> Hi Mohit,

>>>>>> I'm afraid there is nothing like this in Flink yet. As you
>>>>>> mentioned you probably have to manually track the completion of
>>>>>> one job and then trigger execution of the next one.

>>>>>> Best,

>>>>>> Aljoscha 


>>>>>> On Fri, 24 Feb 2017 at 19:16 Mohit Anchlia
>>>>>> <mohitanchlia@gmail.com> wrote:
>>>>>>> Is there a way to connect 2 workflows such that one triggers
>>>>>>> other if certain condition is met? However, the workaround may
>>>>>>> be to insert a notification in a topic to trigger another
>>>>>>> workflow. The problem is that the addSink ends the flow so if
>>>>>>> need to add a trigger after addSink there doesn't seem to be
>>>>>>> good way of sending a notification to a queue that the batch
>>>>>>> processing is complete. Any suggestions? One option could be
>>>>>>> track the progress of a job and on a successful completion add
>>>>>>> notification. Is there such a mechanism available?

View raw message