falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Saktheesh Balaraj (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1618) Unable to run OOZIE (MapReduce) workflow through Falcon Process
Date Thu, 19 Nov 2015 18:29:11 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15014097#comment-15014097
] 

Saktheesh Balaraj commented on FALCON-1618:
-------------------------------------------

In falcon/oozie/src/main/java/org/apache/falcon/oozie/process/ProcessExecutionCoordinatorBuilder.java

The following code which generates dataIn property tags on oozie coordinator file for Storage
type FILESYSTEM doesn't have "coord:" at the beginning.

if (storage.getType() == Storage.TYPE.FILESYSTEM) {
                inputExpr = getELExpression("dataIn('" + input.getName() + "', '" + input.getPartition()
+ "')");
                props.put(input.getName(), inputExpr);
            } else if (storage.getType() == Storage.TYPE.TABLE) {
                inputExpr = "${coord:dataIn('" + input.getName() + "')}";
                propagateCatalogTableProperties(input, (CatalogStorage) storage, props);
            }

Either getELExpression function needs to be modified to prefix "coord:" or can be done at
the above-mentioned line.

> Unable to run OOZIE (MapReduce)  workflow through Falcon Process
> ----------------------------------------------------------------
>
>                 Key: FALCON-1618
>                 URL: https://issues.apache.org/jira/browse/FALCON-1618
>             Project: Falcon
>          Issue Type: Bug
>          Components: oozie
>    Affects Versions: 0.6.1, trunk
>         Environment: HDP 2.3.2
>            Reporter: Saktheesh Balaraj
>
> Error:
> Caused by: org.apache.falcon.FalconException: E1004 : E1004: Expression language evaluation
error, Unable to evaluate :${dataIn('input', 'null')}:
>         at org.apache.falcon.workflow.engine.OozieWorkflowEngine.dryRunInternal(OozieWorkflowEngine.java:234)
> Environment:  HDP2.3.2 Sandbox
> Observation:
> This could be due to wrong property values in OOZIE coordinator file being generated
by Falcon for an OOZIE workflow (MR).
> Current Values:
>                 <property>
>                     <name>falconInPaths</name>
>                     <value>${dataIn('input', 'null')}</value>
>                 </property>
>                 <property>
>                     <name>input</name>
>                     <value>${dataIn('input', 'null')}</value>
>                 </property>
> Expected Values:
> <property>
>                     <name>falconInPaths</name>
>                     <value>coord:{dataIn('input', 'null')}</value>
>                 </property>
>                 <property>
>                     <name>input</name>
>                     <value>coord:{dataIn('input', 'null')}</value>
>                 </property>
> Steps to simulate the problem:
> 1.Install  HDP 2.3.2 Sandbox
> 	2. Enable Falcon through Ambari
> 	3. Configure Feed (input), Feed (output)
> 	4. Configure process with the following configurations
> 		-input (input Feed)
> 		-output (output Feed)
> 	5. Schedule the process through Falcon 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message