falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suhas Vasu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-583) Post processing is broken in current trunk
Date Thu, 14 Aug 2014 11:56:11 GMT

    [ https://issues.apache.org/jira/browse/FALCON-583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096891#comment-14096891
] 

Suhas Vasu commented on FALCON-583:
-----------------------------------

I was testing something in falcon, so just applied the patch & tested.

Its failing with this error:
{noformat}
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.JavaMain], main() threw
exception, org.apache.falcon.FalconException: Error parsing wf args
org.apache.oozie.action.hadoop.JavaMainException: org.apache.falcon.FalconException: Error
parsing wf args
	at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
	at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1438)
	at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: org.apache.falcon.FalconException: Error parsing wf args
	at org.apache.falcon.workflow.WorkflowExecutionContext.create(WorkflowExecutionContext.java:319)
	at org.apache.falcon.workflow.FalconPostProcessing.run(FalconPostProcessing.java:43)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.falcon.workflow.FalconPostProcessing.main(FalconPostProcessing.java:37)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:55)
	... 15 more
Caused by: org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -operation
	at org.apache.commons.cli.Parser.processOption(Parser.java:363)
	at org.apache.commons.cli.Parser.parse(Parser.java:199)
	at org.apache.commons.cli.Parser.parse(Parser.java:119)
	at org.apache.falcon.workflow.WorkflowExecutionContext.getCommand(WorkflowExecutionContext.java:337)
	at org.apache.falcon.workflow.WorkflowExecutionContext.create(WorkflowExecutionContext.java:311)
	... 23 more
{noformat}

The wf args:
{noformat}
>>> Invoking Main class now >>>

Main class        : org.apache.falcon.workflow.FalconPostProcessing
Arguments         :
                    -cluster
                    local2
                    -entityType
                    FEED
                    -entityName
                    infeed-hcat
                    -nominalTime
                    2014-08-11-00-00
                    -operation
                    REPLICATE
                    -workflowId
                    0000007-140814171559062-oozie-oozi-W
                    -runId
                    0
                    -status
                    FAILED
                    -timeStamp
                    2014-08-14-11-50
                    -brokerImplClass
                    org.apache.activemq.ActiveMQConnectionFactory
                    -brokerUrl
                    tcp://localhost:61616
                    -userBrokerImplClass
                    org.apache.activemq.ActiveMQConnectionFactory
                    -userBrokerUrl
                    tcp://localhost:61616?daemon=true
                    -brokerTTL
                    4320
                    -feedNames
                    infeed-hcat
                    -feedInstancePaths
                    hcat://localhost:1109/default/replicated_in_table/Year=2014;Month=08;Date=11;Hour=00;Minute=00
                    -logFile
                    hdfs://localhost:8020/projects/falcon/staging/falcon/workflows/feed/infeed-hcat/logs/instancePaths-2014-08-11-00-00.csv
                    -workflowEngineUrl
                    http://localhost:11000/oozie/
                    -subflowId
                    0000007-140814171559062-oozie-oozi-W
                    -userWorkflowEngine
                    falcon
                    -userWorkflowName
                    replication-policy
                    -userWorkflowVersion
                    0.6-incubating-SNAPSHOT-r30b83d21a2726641435d9290320d5f54f47b3b95
                    -logDir
                    hdfs://localhost:8020/projects/falcon/staging/falcon/workflows/feed/infeed-hcat/logs/job-2014-08-11-00-00/local/
                    -workflowUser
                    suhas.v
                    -falconInputFeeds
                    infeed-hcat
                    -falconInputPaths
                    hcat://localhost:1109/default/in_table/Year=2014;Month=08;Date=11;Hour=00;Minute=00


<<< Invocation of Main class completed <<<
{noformat}

> Post processing is broken in current trunk
> ------------------------------------------
>
>                 Key: FALCON-583
>                 URL: https://issues.apache.org/jira/browse/FALCON-583
>             Project: Falcon
>          Issue Type: Bug
>    Affects Versions: 0.6
>            Reporter: Suhas Vasu
>            Assignee: Venkatesh Seetharam
>            Priority: Blocker
>             Fix For: 0.6
>
>         Attachments: FALCON-583.patch
>
>
> Currently falcon post-processing is broken.
> Checked it for the following instances:
> 1. A feed with hcat-replication
> 2. A MR process that reads from hadoop fs and writes to hadoop fs
> Both of the actions failed at x@succeeded-post-processing with the error 
> {noformat}
> Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.JavaMain], main()
threw exception, java.lang.NullPointerException
> org.apache.oozie.action.hadoop.JavaMainException: java.lang.NullPointerException
> 	at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
> 	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
> 	at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1438)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:262)
> Caused by: java.lang.NullPointerException
> 	at org.apache.commons.cli.GnuParser.flatten(GnuParser.java:68)
> 	at org.apache.commons.cli.Parser.parse(Parser.java:161)
> 	at org.apache.commons.cli.Parser.parse(Parser.java:119)
> 	at org.apache.falcon.workflow.WorkflowExecutionContext.getCommand(WorkflowExecutionContext.java:337)
> 	at org.apache.falcon.workflow.WorkflowExecutionContext.create(WorkflowExecutionContext.java:311)
> 	at org.apache.falcon.workflow.FalconPostProcessing.run(FalconPostProcessing.java:43)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> 	at org.apache.falcon.workflow.FalconPostProcessing.main(FalconPostProcessing.java:37)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:55)
> 	... 15 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message