Mailing-List: contact dev-help@falcon.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@falcon.incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of shwetha.gs@inmobi.com
 designates 209.85.192.181 as permitted sender)
From: Shwetha GS <shwetha.gs@inmobi.com>
Mime-Version: 1.0 (1.0)
Subject: Re: Using properties in processes
Message-Id: <CEBBE6FD-46F5-4D6F-9068-B6E3F017F0F0@inmobi.com>
Date: Thu, 10 Jul 2014 20:20:53 +0530
References: 
 <CAANBrOWsRjUAGS9Oj3RGXTAc=dYqR-n7MFit_jFzDdQ7eLra2A@mail.gmail.com>
 <BLU179-W853F991284F79097B8DB81A40F0@phx.gbl>
 <CAFfn_RpuJszio--eFHWw6QM1Gh+swi0VvoV+CnDKnj4cs=A_FQ@mail.gmail.com>
 <CAANBrOWeT+U_f5vhANi_iCA8cGFFw1Uv-L-W_mAXEg-FpnD38w@mail.gmail.com>
In-Reply-To: 
 <CAANBrOWeT+U_f5vhANi_iCA8cGFFw1Uv-L-W_mAXEg-FpnD38w@mail.gmail.com>
To: "dev@falcon.incubator.apache.org" <dev@falcon.incubator.apache.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Falcon internally uses oozie. Looks like oozie doesn't support this.=20

Will check oozie tomorrow and see if it can be fixed in oozie

Sent from my iPhone

> On Jul 10, 2014, at 5:59 PM, Josh Clum <joshclum@gmail.com> wrote:
>=20
> It took me forever, but I finally found this error in the bundle logs. If=
 I
> reference a property in my process xml file,
>=20
> ie <property name=3D"indexSelect" value=3D"Select .... where
> ${falcon_input_partition_filter_hive}" />
>=20
> then I get:
>=20
> at
> org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java=
:696)
> at
> org.apache.oozie.command.coord.CoordSubmitXCommand.resolveTagContents(Coo=
rdSubmitXCommand.java:895)
> ... 10 more
> Caused by: javax.servlet.jsp.el.ELException: variable
> [falcon_input_partition_filter_hive] cannot be resolved
> at
> org.apache.oozie.util.ELEvaluator$Context.resolveVariable(ELEvaluator.jav=
a:106)
> at org.apache.commons.el.NamedValue.evaluate(NamedValue.java:124)
> at
> org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114=
)
> at
> org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluato=
rImpl.java:274)
> at
> org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluato=
rImpl.java:190)
> at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203)
> at
> org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java=
:687)
> ... 11 more
>=20
> If I try and use the oozie el functions in my process xml
>=20
> ie <property name=3D"indexSelect" value=3D"Select .... where
> ${wf:conf('falcon_input_partition_filter_hive')}" />
>=20
> then I get:
>=20
> at
> org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java=
:696)
> at
> org.apache.oozie.command.coord.CoordSubmitXCommand.resolveTagContents(Coo=
rdSubmitXCommand.java:895)
> ... 10 more
> Caused by: javax.servlet.jsp.el.ELException: No function is mapped to the
> name "wf:conf"
> at org.apache.commons.el.Logger.logError(Logger.java:481)
> at org.apache.commons.el.Logger.logError(Logger.java:498)
> at org.apache.commons.el.Logger.logError(Logger.java:525)
> at
> org.apache.commons.el.FunctionInvocation.evaluate(FunctionInvocation.java=
:150)
> at
> org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114=
)
> at
> org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluato=
rImpl.java:274)
> at
> org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluato=
rImpl.java:190)
> at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203)
> at
> org.apache.oozie.coord.CoordELFunctions.evalAndWrap(CoordELFunctions.java=
:687)
> ... 11 more
>=20
> I also tried defining a property earlier in my process xml and then using
> it later with no luck.
>=20
> ie
>=20
> <properties>
>=20
> ...
>=20
> <property name=3D"foo" value=3D":)" />
> ...
> <property name=3D"bar" value=3D"${foo}" />
>=20
> ...
>=20
> </properties>
>=20
> which still gives me the ELException: variable
> [falcon_input_partition_filter_hive] cannot be resolved.
>=20
> So it would seem that there is no way to access configurations within the
> process xml itself. Is this true?
>=20
> Thanks,
>=20
> Josh
>=20
>=20
> On Wed, Jul 9, 2014 at 11:05 AM, Seetharam Venkatesh <
> venkatesh@innerzeal.com> wrote:
>=20
>> Josh,
>>=20
>> These variables are exposed to user. The convention is:
>>=20
>> falcon_${input_feed_name}_partition_filter_pig
>> falcon_${input_feed_name}_partition_filter_hive
>> falcon_${input_feed_name}_partition_filter_java
>> falcon_${input_feed_name}_datain_partitions_hive
>>=20
>> You should be able to use any of these.
>>=20
>> If you are having issues with oozie starting the jobs, its good to look =
at
>> oozie.log and see if there an error with any EL.
>>=20
>> Thanks!
>>=20
>>=20
>> On Wed, Jul 9, 2014 at 12:57 PM, Srikanth Sundarrajan <sriksun@hotmail.c=
om
>> wrote:
>>=20
>>> In general it is not a good practice to refer to Falcon internal
>>> variables, but this might be a public one. Will let @Venkatesh confirm
>>> this. If it is indeed public, this ought to work, but this may also lea=
d
>> to
>>> the possibility of cyclic referencing of properties. A peek into the
>> Oozie
>>> logs might give ideas on what might be the reason for coord not startin=
g.
>>>=20
>>> Regards
>>> Srikanth Sundarrajan
>>>=20
>>>=20
>>> ----------------------------------------
>>>> From: joshclum@gmail.com
>>>> Date: Tue, 8 Jul 2014 15:47:10 -0400
>>>> Subject: Using properties in processes
>>>> To: dev@falcon.incubator.apache.org
>>>>=20
>>>> Hi,
>>>>=20
>>>> Is it possible to reference properties like
>>> falcon_input_partition_filter_hive
>>>> inside of the process xml definition itself?
>>>>=20
>>>> For instance:
>>>>=20
>>>> <properties>
>>>> ...
>>>> <property name=3D"indexSelect" value=3D"Select .... where
>>>> ${falcon_input_partition_filter_hive}" />
>>>>=20
>>>> ...
>>>>=20
>>>> </properties>
>>>>=20
>>>>=20
>>>> When I try to do this, I don't get any errors, but my process never
>>> starts
>>>> up either. Looked in the falcon logs with no luck. Ideas?
>>>>=20
>>>> Thanks,
>>>> Josh
>>=20
>>=20
>>=20
>> --
>> Regards,
>> Venkatesh
>>=20
>> =E2=80=9CPerfection (in design) is achieved not when there is nothing mo=
re to add,
>> but rather when there is nothing more to take away.=E2=80=9D
>> - Antoine de Saint-Exup=C3=A9ry
>>=20

--=20
_____________________________________________________________
The information contained in this communication is intended solely for the=
=20
use of the individual or entity to whom it is addressed and others=20
authorized to receive it. It may contain confidential or legally privileged=
=20
information. If you are not the intended recipient you are hereby notified=
=20
that any disclosure, copying, distribution or taking any action in reliance=
=20
on the contents of this information is strictly prohibited and may be=20
unlawful. If you have received this communication in error, please notify=
=20
us immediately by responding to this email and then delete it from your=20
system. The firm is neither liable for the proper and complete transmission=
=20
of the information contained in this communication nor for any delay in its=
=20
receipt.