incubator-oozie-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Islam <misla...@yahoo.com>
Subject Re: Oozie workflow split in files?
Date Thu, 05 Apr 2012 06:30:14 GMT
Hi Deepika,
Yes you can write a WF application with multiple workflow xml files in the same directory
say w1.xml, w2.xml, .., w9.xml=A0 and w_combo.xml.
In w_combo.xml, you can execute sub-wf actions for each wf?.xmlin any sequence (such as forking).

Do you see real value to trigger multiple WFs from one coordinator? The question is in what
sequence you want to launch those WFs? The reason, we don't want to support another DAG engine
like concept at coordinator level.

Regards,
Mohammad


________________________________
From: Deepika Khera <dkhera@lyris.com>
To: "oozie-users@incubator.apache.org" <oozie-users@incubator.apache.org> 
Sent: Wednesday, April 4, 2012 5:01 PM
Subject: Re: Oozie workflow split in files?

Thanks Max. Yeah, seems like a reasonable way to achieve this.
I can split my cumbersome "hourly" workflow, into separate workflows
applications and then a main workflow can bundle them up as subworkflows
and trigger them all at once(through a common coordinator maybe).
I tried it out (though without the coordinator at this point) and it
seems to work fine.
There is an overhead of deploying these as separate workflows rather
than having them all together in a single workflow application, but that
should be ok (unless there is another way to do this that we are unaware
of)

Thanks!
Deepika



On Wed, 2012-04-04 at 16:16 -0700, Maxime Petazzoni wrote:
> Deepika,
> 
> * Deepika Khera <dkhera@lyris.com> [2012-04-04 16:09:43]:
> 
> > I have about 70 jobs to run as a part of an hourly workflow. I was
> > wondering if it will be possible to split these into separate workflow
> > files (like each sub-workflow in a separate file), so it is not one
> > large file describing all the workflow jobs.
> > Is there a way to achieve something like this?
> > Currently , I can find only examples of defining sub-workflows in the
> > same file.
> 
> Are these jobs independant? You can split the independant blocks into
> distincts workflows and coordinators, and submit them all at once as a
> bundle maybe?
> 
> Note that I'm not sure this is really the "best practice" for this
> situation. I'm still trying to figure a lot of things out with Oozie
> myself :)
> 
> /Max

Mime
View raw message