incubator-oozie-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deepika Khera <dkh...@lyris.com>
Subject Re: Oozie workflow split in files?
Date Thu, 05 Apr 2012 18:04:37 GMT
These jobs are actually a part of a single workflow, though they can be
logically grouped into categories. Since the number of jobs is quite
big, I wanted to split them into separate files for simplicity.

In sub-workflow schema I only see the possibility of elements -
app-path, propagate-configuration and configuration . 
Could you please post a small sample of how these multiple workflow
files could be embedded into a single workflow? 

Oh and as Max pointed out I am not aware of a way to rename workflow.xml
to some other name. Is there a configuration that can be used for this
in oozie-site.xml?

Thanks,
Deepika

On Wed, 2012-04-04 at 23:30 -0700, Mohammad Islam wrote:
> Hi Deepika,
> Yes you can write a WF application with multiple workflow xml files in the same directory
say w1.xml, w2.xml, .., w9.xml=A0 and w_combo.xml.
> In w_combo.xml, you can execute sub-wf actions for each wf?.xmlin any sequence (such
as forking).
> 
> Do you see real value to trigger multiple WFs from one coordinator? The question is in
what sequence you want to launch those WFs? The reason, we don't want to support another DAG
engine like concept at coordinator level.
> 
> Regards,
> Mohammad
> 
> 
> ________________________________
> From: Deepika Khera <dkhera@lyris.com>
> To: "oozie-users@incubator.apache.org" <oozie-users@incubator.apache.org> 
> Sent: Wednesday, April 4, 2012 5:01 PM
> Subject: Re: Oozie workflow split in files?
> 
> Thanks Max. Yeah, seems like a reasonable way to achieve this.
> I can split my cumbersome "hourly" workflow, into separate workflows
> applications and then a main workflow can bundle them up as subworkflows
> and trigger them all at once(through a common coordinator maybe).
> I tried it out (though without the coordinator at this point) and it
> seems to work fine.
> There is an overhead of deploying these as separate workflows rather
> than having them all together in a single workflow application, but that
> should be ok (unless there is another way to do this that we are unaware
> of)
> 
> Thanks!
> Deepika
> 
> 
> 
> On Wed, 2012-04-04 at 16:16 -0700, Maxime Petazzoni wrote:
> > Deepika,
> > 
> > * Deepika Khera <dkhera@lyris.com> [2012-04-04 16:09:43]:
> > 
> > > I have about 70 jobs to run as a part of an hourly workflow. I was
> > > wondering if it will be possible to split these into separate workflow
> > > files (like each sub-workflow in a separate file), so it is not one
> > > large file describing all the workflow jobs.
> > > Is there a way to achieve something like this?
> > > Currently , I can find only examples of defining sub-workflows in the
> > > same file.
> > 
> > Are these jobs independant? You can split the independant blocks into
> > distincts workflows and coordinators, and submit them all at once as a
> > bundle maybe?
> > 
> > Note that I'm not sure this is really the "best practice" for this
> > situation. I'm still trying to figure a lot of things out with Oozie
> > myself :)
> > 
> > /Max



Mime
View raw message