flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Re: Job scheduling
Date Thu, 11 Sep 2014 14:29:53 GMT
Of course with Flink I could in principle execute almost everything with a
single Job but, in general, I could write 2 different jobs and decide from
time to time when the second should be run.
That's why also metheor scripts are very useful :)
>From what I know there was a scheduler in Stratosphere that was using
RabbitMQ, right?

I would like to avoid to run linux commands and instead use some REST
interface to trigger or schedule jobs.


On Thu, Sep 11, 2014 at 4:07 PM, Fabian Hueske <fhueske@apache.org> wrote:

> Hi Flavio,
> what exactly do you mean by scheduling?
> Do you want to run a job in regular intervals or execute a complex
> workflow?
> Oozie is primarily used to orchestrate the execution of MapReduce
> workflows. Since, MR is a rather inflexible programming model, complex
> tasks need to split up into multiple dependent jobs that are executed once
> their predecessors have finished. Oozie orchestrates this execution.
> In Flink, you can build a complex analysis flow as a single program and
> execute it. Hence, there is no need for a workflow scheduler such as Oozie.
> If you want to run a job in regular intervals, you can configure a cron
> job, that starts executes the CLI client or implement a Java or Scala
> program that submits jobs a certain points in time.
> Best, Fabian
> 2014-09-11 15:36 GMT+02:00 Flavio Pompermaier <pompermaier@okkam.it>:
>> Hi to all,
>> I'd like to know if there's an example of how to schedule a Job in Flink.
>> Do we still need something like Oozie or Quartz or we can avoid them?
>> Best,
>> Flavio

View raw message