falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Saktheesh Balaraj (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1618) Unable to run OOZIE (MapReduce) workflow through Falcon Process
Date Thu, 26 Nov 2015 11:40:11 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028609#comment-15028609
] 

Saktheesh Balaraj commented on FALCON-1618:
-------------------------------------------

It's working after adding the following parameters in oozie-site.xml. Note: HDP 2.3 works
without setting these parameters, but 2.3.2 requires.

oozie.service.ELService.ext.functions.coord-job-submit-instances

                now=org.apache.oozie.extensions.OozieELExtensions#ph1_now_echo,

      today=org.apache.oozie.extensions.OozieELExtensions#ph1_today_echo,

      yesterday=org.apache.oozie.extensions.OozieELExtensions#ph1_yesterday_echo,

      currentWeek=org.apache.oozie.extensions.OozieELExtensions#ph1_currentWeek_echo,

      lastWeek=org.apache.oozie.extensions.OozieELExtensions#ph1_lastWeek_echo,

      currentMonth=org.apache.oozie.extensions.OozieELExtensions#ph1_currentMonth_echo,

      lastMonth=org.apache.oozie.extensions.OozieELExtensions#ph1_lastMonth_echo,

      currentYear=org.apache.oozie.extensions.OozieELExtensions#ph1_currentYear_echo,

      lastYear=org.apache.oozie.extensions.OozieELExtensions#ph1_lastYear_echo,

      formatTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_formatTime_echo,

      latest=org.apache.oozie.coord.CoordELFunctions#ph2_coord_latest_echo,

      future=org.apache.oozie.coord.CoordELFunctions#ph2_coord_future_echo

oozie.service.ELService.ext.functions.coord-action-create-inst
                now=org.apache.oozie.extensions.OozieELExtensions#ph2_now_inst,

      today=org.apache.oozie.extensions.OozieELExtensions#ph2_today_inst,

      yesterday=org.apache.oozie.extensions.OozieELExtensions#ph2_yesterday_inst,

      currentWeek=org.apache.oozie.extensions.OozieELExtensions#ph2_currentWeek_inst,

      lastWeek=org.apache.oozie.extensions.OozieELExtensions#ph2_lastWeek_inst,

      currentMonth=org.apache.oozie.extensions.OozieELExtensions#ph2_currentMonth_inst,

      lastMonth=org.apache.oozie.extensions.OozieELExtensions#ph2_lastMonth_inst,

      currentYear=org.apache.oozie.extensions.OozieELExtensions#ph2_currentYear_inst,

      lastYear=org.apache.oozie.extensions.OozieELExtensions#ph2_lastYear_inst,

      latest=org.apache.oozie.coord.CoordELFunctions#ph2_coord_latest_echo,

      future=org.apache.oozie.coord.CoordELFunctions#ph2_coord_future_echo,

      formatTime=org.apache.oozie.coord.CoordELFunctions#ph2_coord_formatTime,

      user=org.apache.oozie.coord.CoordELFunctions#coord_user

oozie.service.ELService.ext.functions.coord-action-create
                now=org.apache.oozie.extensions.OozieELExtensions#ph2_now,

      today=org.apache.oozie.extensions.OozieELExtensions#ph2_today,

      yesterday=org.apache.oozie.extensions.OozieELExtensions#ph2_yesterday,

      currentWeek=org.apache.oozie.extensions.OozieELExtensions#ph2_currentWeek,

      lastWeek=org.apache.oozie.extensions.OozieELExtensions#ph2_lastWeek,

      currentMonth=org.apache.oozie.extensions.OozieELExtensions#ph2_currentMonth,

      lastMonth=org.apache.oozie.extensions.OozieELExtensions#ph2_lastMonth,

      currentYear=org.apache.oozie.extensions.OozieELExtensions#ph2_currentYear,

      lastYear=org.apache.oozie.extensions.OozieELExtensions#ph2_lastYear,

      latest=org.apache.oozie.coord.CoordELFunctions#ph2_coord_latest_echo,

      future=org.apache.oozie.coord.CoordELFunctions#ph2_coord_future_echo,

      formatTime=org.apache.oozie.coord.CoordELFunctions#ph2_coord_formatTime,

      user=org.apache.oozie.coord.CoordELFunctions#coord_user

oozie.service.ELService.ext.functions.coord-job-submit-data

                now=org.apache.oozie.extensions.OozieELExtensions#ph1_now_echo,

      today=org.apache.oozie.extensions.OozieELExtensions#ph1_today_echo,

      yesterday=org.apache.oozie.extensions.OozieELExtensions#ph1_yesterday_echo,

      currentWeek=org.apache.oozie.extensions.OozieELExtensions#ph1_currentWeek_echo,

      lastWeek=org.apache.oozie.extensions.OozieELExtensions#ph1_lastWeek_echo,

      currentMonth=org.apache.oozie.extensions.OozieELExtensions#ph1_currentMonth_echo,

      lastMonth=org.apache.oozie.extensions.OozieELExtensions#ph1_lastMonth_echo,

      currentYear=org.apache.oozie.extensions.OozieELExtensions#ph1_currentYear_echo,

      lastYear=org.apache.oozie.extensions.OozieELExtensions#ph1_lastYear_echo,

      dataIn=org.apache.oozie.extensions.OozieELExtensions#ph1_dataIn_echo,

      instanceTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_nominalTime_echo_wrap,

      formatTime=org.apache.oozie.coord.CoordELFunctions#ph1_coord_formatTime_echo,

      dateOffset=org.apache.oozie.coord.CoordELFunctions#ph1_coord_dateOffset_echo,

      user=org.apache.oozie.coord.CoordELFunctions#coord_user

oozie.service.ELService.ext.functions.coord-action-start
                now=org.apache.oozie.extensions.OozieELExtensions#ph2_now,

      today=org.apache.oozie.extensions.OozieELExtensions#ph2_today,

      yesterday=org.apache.oozie.extensions.OozieELExtensions#ph2_yesterday,

      currentWeek=org.apache.oozie.extensions.OozieELExtensions#ph2_currentWeek,

      lastWeek=org.apache.oozie.extensions.OozieELExtensions#ph2_lastWeek,

      currentMonth=org.apache.oozie.extensions.OozieELExtensions#ph2_currentMonth,

      lastMonth=org.apache.oozie.extensions.OozieELExtensions#ph2_lastMonth,

      currentYear=org.apache.oozie.extensions.OozieELExtensions#ph2_currentYear,

      lastYear=org.apache.oozie.extensions.OozieELExtensions#ph2_lastYear,

      latest=org.apache.oozie.coord.CoordELFunctions#ph3_coord_latest,

      future=org.apache.oozie.coord.CoordELFunctions#ph3_coord_future,

      dataIn=org.apache.oozie.extensions.OozieELExtensions#ph3_dataIn,

      instanceTime=org.apache.oozie.coord.CoordELFunctions#ph3_coord_nominalTime,

      dateOffset=org.apache.oozie.coord.CoordELFunctions#ph3_coord_dateOffset,

      formatTime=org.apache.oozie.coord.CoordELFunctions#ph3_coord_formatTime,

      user=org.apache.oozie.coord.CoordELFunctions#coord_user

> Unable to run OOZIE (MapReduce)  workflow through Falcon Process
> ----------------------------------------------------------------
>
>                 Key: FALCON-1618
>                 URL: https://issues.apache.org/jira/browse/FALCON-1618
>             Project: Falcon
>          Issue Type: Bug
>          Components: oozie
>    Affects Versions: 0.6.1, trunk
>         Environment: HDP 2.3.2
>            Reporter: Saktheesh Balaraj
>            Assignee: Peeyush Bishnoi
>            Priority: Blocker
>         Attachments: SampleProcess.xml, Simplefeedin.xml, Simplefeedout.xml, workflow.xml
>
>
> Error:
> ~~~~
> Caused by: org.apache.falcon.FalconException: E1004 : E1004: Expression language evaluation
error, Unable to evaluate :${dataIn('input', 'null')}:
>         at org.apache.falcon.workflow.engine.OozieWorkflowEngine.dryRunInternal(OozieWorkflowEngine.java:234)
> Environment:  HDP2.3.2 Sandbox
> ~~~~~~~~~~
> Observation:
> ~~~~~~~~~~
> This could be due to wrong property values in OOZIE coordinator file being generated
by Falcon for an OOZIE workflow (MR).
> Current Values:
>                 <property>
>                     <name>falconInPaths</name>
>                     <value>${dataIn('input', 'null')}</value>
>                 </property>
>                 <property>
>                     <name>input</name>
>                     <value>${dataIn('input', 'null')}</value>
>                 </property>
> Expected Values:
> ~~~~~~~~~~~~~
> <property>
>                     <name>falconInPaths</name>
>                     <value>coord:{dataIn('input', 'null')}</value>
>                 </property>
>                 <property>
>                     <name>input</name>
>                     <value>coord:{dataIn('input', 'null')}</value>
>                 </property>
> Steps to simulate the problem:
> ~~~~~~~~~~~~~~~~~~~~~~~~
> 1.Install  HDP 2.3.2 Sandbox
> 	2. Enable Falcon through Ambari
> 	3. Configure Feed (input), Feed (output)
> 	4. Configure process with the following configurations
> 		-input (input Feed)
> 		-output (output Feed)
> 	5. Schedule the process through Falcon 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message