falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pallavi Rao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1573) Supply user-defined properties to Oozie workflows during schedule
Date Wed, 04 Nov 2015 04:53:27 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988873#comment-14988873
] 

Pallavi Rao commented on FALCON-1573:
-------------------------------------

[~ddcprg], totally appreciate your contribution. I agree that your patch keeps it simple.
However, personally, I prefer the second approach you proposed - that of using a namespace
for user properties. A few reasons for my preference:
1. We have been wanting to distinguish between falcon system properties(falcon.system) and
user properties(falcon.user) in the entity definition for a while now. This also enables users
to override the values at schedule time, that is more intuitive for a user.
2. Having namespace allows Falcon to consume what is relevant to it and pass on to downstream
engines what is not. For example, when user says falcon.x.y, Falcon will intercept it. If
it is oozie.x.y, it will just pass it along to Oozie.
3. It fits well with our uber feature where we want to support definition time, deployment
time and runtime properties.
4. If we do not use namespace when we introduce the feature, it becomes hard to introduce
later because it would break backward compatibility.

Thoughts are more than welcome.

> Supply user-defined properties to Oozie workflows during schedule
> -----------------------------------------------------------------
>
>                 Key: FALCON-1573
>                 URL: https://issues.apache.org/jira/browse/FALCON-1573
>             Project: Falcon
>          Issue Type: New Feature
>          Components: oozie
>    Affects Versions: trunk
>            Reporter: Daniel del Castillo
>            Priority: Minor
>             Fix For: trunk
>
>         Attachments: FALCON-1573-v1.patch
>
>
> The work in FALCON-1434 added the ability to specify properties for the purpose of enabling
scheduler selection. However, this could be more generally useful as a means for users to
submit custom properties that will be visible in Oozie workflows.
> This will enable users to reference properties in the entity XML itself and passing these
properties down to workflows. For example, a process scheduled with the command:
> {code}
> bin/falcon entity -schedule -type process -name helloFalconProcess -properties custom.property:HEY
> {code}
> can reference the property _custom.property_ in the process XML as
> {code:xml}
> <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
> <process name="helloFalconProcess" xmlns="uri:falcon:process:0.1">
>   ...
>   <properties>
>     ...
>     <property name="customProperty" value="${coord:conf('custom.property')}"/>
>     ...
>   </properties>
>   ...
> </process>
> {code}
> An Oozie workflow can then easily make use of such property:
> {code:xml}
> <workflow-app xmlns="uri:oozie:workflow:0.4" name="hello-falcon-wf">
>   ...
>     <java>
>       ...
>       <arg>--customValue=${customProperty}</arg>
>       ...
>     </java>
>     ...
> </workflow-app>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message