falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Clum <joshc...@gmail.com>
Subject Re: Using properties in processes
Date Thu, 10 Jul 2014 12:29:46 GMT
It took me forever, but I finally found this error in the bundle logs. If I
reference a property in my process xml file,

ie <property name="indexSelect" value="Select .... where
${falcon_input_partition_filter_hive}" />

then I get:

at
org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:696)
 at
org.apache.oozie.command.coord.CoordSubmitXCommand.resolveTagContents(CoordSubmitXCommand.java:895)
... 10 more
Caused by: javax.servlet.jsp.el.ELException: variable
[falcon_input_partition_filter_hive] cannot be resolved
at
org.apache.oozie.util.ELEvaluator$Context.resolveVariable(ELEvaluator.java:106)
 at org.apache.commons.el.NamedValue.evaluate(NamedValue.java:124)
at
org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114)
 at
org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274)
at
org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
 at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203)
at
org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:687)
 ... 11 more

If I try and use the oozie el functions in my process xml

ie <property name="indexSelect" value="Select .... where
${wf:conf('falcon_input_partition_filter_hive')}" />

then I get:

at
org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:696)
 at
org.apache.oozie.command.coord.CoordSubmitXCommand.resolveTagContents(CoordSubmitXCommand.java:895)
... 10 more
Caused by: javax.servlet.jsp.el.ELException: No function is mapped to the
name "wf:conf"
at org.apache.commons.el.Logger.logError(Logger.java:481)
 at org.apache.commons.el.Logger.logError(Logger.java:498)
at org.apache.commons.el.Logger.logError(Logger.java:525)
 at
org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java:150)
at
org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114)
 at
org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274)
at
org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
 at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203)
at
org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java:687)
 ... 11 more

I also tried defining a property earlier in my process xml and then using
it later with no luck.

ie

<properties>

...

<property name="foo" value=":)" />
...
<property name="bar" value="${foo}" />

...

</properties>

which still gives me the ELException: variable
[falcon_input_partition_filter_hive] cannot be resolved.

So it would seem that there is no way to access configurations within the
process xml itself. Is this true?

Thanks,

Josh


On Wed, Jul 9, 2014 at 11:05 AM, Seetharam Venkatesh <
venkatesh@innerzeal.com> wrote:

> Josh,
>
> These variables are exposed to user. The convention is:
>
> falcon_${input_feed_name}_partition_filter_pig
> falcon_${input_feed_name}_partition_filter_hive
> falcon_${input_feed_name}_partition_filter_java
> falcon_${input_feed_name}_datain_partitions_hive
>
> You should be able to use any of these.
>
> If you are having issues with oozie starting the jobs, its good to look at
> oozie.log and see if there an error with any EL.
>
> Thanks!
>
>
> On Wed, Jul 9, 2014 at 12:57 PM, Srikanth Sundarrajan <sriksun@hotmail.com
> >
> wrote:
>
> > In general it is not a good practice to refer to Falcon internal
> > variables, but this might be a public one. Will let @Venkatesh confirm
> > this. If it is indeed public, this ought to work, but this may also lead
> to
> > the possibility of cyclic referencing of properties. A peek into the
> Oozie
> > logs might give ideas on what might be the reason for coord not starting.
> >
> > Regards
> > Srikanth Sundarrajan
> >
> >
> > ----------------------------------------
> > > From: joshclum@gmail.com
> > > Date: Tue, 8 Jul 2014 15:47:10 -0400
> > > Subject: Using properties in processes
> > > To: dev@falcon.incubator.apache.org
> > >
> > > Hi,
> > >
> > > Is it possible to reference properties like
> > falcon_input_partition_filter_hive
> > > inside of the process xml definition itself?
> > >
> > > For instance:
> > >
> > > <properties>
> > > ...
> > > <property name="indexSelect" value="Select .... where
> > > ${falcon_input_partition_filter_hive}" />
> > >
> > > ...
> > >
> > > </properties>
> > >
> > >
> > > When I try to do this, I don't get any errors, but my process never
> > starts
> > > up either. Looked in the falcon logs with no luck. Ideas?
> > >
> > > Thanks,
> > > Josh
> >
> >
>
>
>
> --
> Regards,
> Venkatesh
>
> “Perfection (in design) is achieved not when there is nothing more to add,
> but rather when there is nothing more to take away.”
> - Antoine de Saint-Exupéry
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message