incubator-oozie-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Angelo K. Huang" <ange...@apache.org>
Subject Re: Oozie coordinator xml data input dependency
Date Mon, 03 Oct 2011 03:30:47 GMT
Could you post your coordinator.xml so we can easily understand the context?
Do you have _SUCCESS at both directories? Oozie use that for checking
dependencies.

http://yahoo.github.com/oozie/releases/3.1.0/CoordinatorFunctionalSpec.html#a5.1._Synchronous_Datasets

You can also set <done-flag></done-flag> so Oozie looks for the existence of
the directory itself.

Thanks,
Angelo

On Sun, Oct 2, 2011 at 8:06 PM, Deepak Reddy <dreddy@couponsinc.com> wrote:

> My job A writes the output as data/user/dreddy/input/yyyyMMddHHmm (5min
> freq)
>
> Each instance of job B (5min freq) should look for corresponding job A
> directory as above
>
> So in my <datasets> tag I have
> hdfs://${namenodeval}/data/user/dreddy/input/${YEAR}${MONTH}${DAY}${HOUR}${MINUTE}
>
> Where initial instance of job A is same as startdate of job B
>
> Also in the <input-events> section I have the start and end instance as
> ${coord:current(0)} (since job B is dependent on only one dataset of job A)
>
> I created 2 instances of job A as /data/user/dreddy/input/201110010545 and
> /data/user/dreddy/input/201110010550
>
> And when I ran the XML I see that the coord jobs are in Waiting state all
> the time (doesn't launch any workflow jobs)
>
> Am I missing something in my coord XML ?
>
> Thanks,
> DR
>
> -----Original Message-----
> From: Angelo K. Huang [mailto:angelohuang@gmail.com]
> Sent: Sunday, October 02, 2011 5:44 PM
> To: oozie-users@incubator.apache.org
> Subject: Re: Oozie coordinator xml data input dependency
>
> If you want to specify the time stamp at url-template, there are other
> variable that you can use: For example, ${YEAR}${MONTH}${DAY}${HOUR}.
> The coord:nominalTime
> is not available at url-templates. Could you try use YEAR, MONTH, etc at
> your coordinator.xml's dataset instead? Let us know if it works for you.
>
>
> http://yahoo.github.com/oozie/releases/3.1.0/CoordinatorFunctionalSpec.html#a5.1._Synchronous_Datasets
>
> Thanks,
> Angelo
>
> On Sun, Oct 2, 2011 at 4:37 PM, Deepak Reddy <dreddy@couponsinc.com>
> wrote:
>
> > Hi,
> >
> > In my coordinator.xml I have a data input dependency
> >
> > When I try to use a coord:formatTime EL function in the <uri-template>
> > and try to run the job it gives me
> >
> > Unable to evaluate
> > :hdfs://${namenodeval}/data/user/input/${coord:formatTime(coord:nomina
> > lTime(),
> > 'yyyyMMddHHmm')}: ], java.lang.Exception: Unable to evaluate
> > :hdfs://${namenodeval}:54310/data/user/dreddy/input/${coord:formatTime
> > (coord:nominalTime(),
> > 'yyyyMMddHHmm')}
> >
> > Cant I use EL functions in the <datasets> part of the XML (the EL
> > function works for me in the <configuration> part)
> >
> > Removing this EL function works for me but I would like to specify a
> > batchdate in a particular format (this batchdate is part of the
> > filepath)
> >
> > Thanks,
> > DR
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message