falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sowmya Ramesh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-789) Post processing is broken for Feeds
Date Wed, 08 Oct 2014 23:20:36 GMT

    [ https://issues.apache.org/jira/browse/FALCON-789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14164369#comment-14164369
] 

Sowmya Ramesh commented on FALCON-789:
--------------------------------------

Patch looks good. I added additional unit tests for WorkflowExecutionContext#getFilepath.Please
find the patch attached.

> Post processing is broken for Feeds
> -----------------------------------
>
>                 Key: FALCON-789
>                 URL: https://issues.apache.org/jira/browse/FALCON-789
>             Project: Falcon
>          Issue Type: Bug
>          Components: messaging
>    Affects Versions: 0.6
>            Reporter: Sowmya Ramesh
>            Assignee: Venkatesh Seetharam
>            Priority: Blocker
>              Labels: PostProcessing
>             Fix For: 0.6
>
>         Attachments: FALCON-789.patch
>
>
> PostProcessing is broken for Feeds. Found this issue for replication and retention policies.
> Looks like bug was introduced by [FALCON-773|https://issues.apache.org/jira/browse/FALCON-773].
> {noformat}
> Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.JavaMain], main()
threw exception, java.lang.IllegalArgumentException: Pathname /context/hdfs:/falcon-distcp-1.cs1cloud.internal:8020/apps/falcon/bcpCluster/staging/falcon/workflows/feed/Falcon-20-test-Feed/logs/job-2014-10-08-03-29/primaryCluster/Falcon-20-test-Feed-wf-post-exec-context.json
from /context/hdfs:/falcon-distcp-1.cs1cloud.internal:8020/apps/falcon/bcpCluster/staging/falcon/workflows/feed/Falcon-20-test-Feed/logs/job-2014-10-08-03-29/primaryCluster/Falcon-20-test-Feed-wf-post-exec-context.json
is not a valid DFS filename.
> org.apache.oozie.action.hadoop.JavaMainException: java.lang.IllegalArgumentException:
Pathname /context/hdfs:/falcon-distcp-1.cs1cloud.internal:8020/apps/falcon/bcpCluster/staging/falcon/workflows/feed/Falcon-20-test-Feed/logs/job-2014-10-08-03-29/primaryCluster/Falcon-20-test-Feed-wf-post-exec-context.json
from /context/hdfs:/falcon-distcp-1.cs1cloud.internal:8020/apps/falcon/bcpCluster/staging/falcon/workflows/feed/Falcon-20-test-Feed/logs/job-2014-10-08-03-29/primaryCluster/Falcon-20-test-Feed-wf-post-exec-context.json
is not a valid DFS filename.
> 	at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:58)
> 	at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
> 	at org.apache.oozie.action.hadoop.JavaMain.main(JavaMain.java:36)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
> 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> Caused by: java.lang.IllegalArgumentException: Pathname /context/hdfs:/falcon-distcp-1.cs1cloud.internal:8020/apps/falcon/bcpCluster/staging/falcon/workflows/feed/Falcon-20-test-Feed/logs/job-2014-10-08-03-29/primaryCluster/Falcon-20-test-Feed-wf-post-exec-context.json
from /context/hdfs:/falcon-distcp-1.cs1cloud.internal:8020/apps/falcon/bcpCluster/staging/falcon/workflows/feed/Falcon-20-test-Feed/logs/job-2014-10-08-03-29/primaryCluster/Falcon-20-test-Feed-wf-post-exec-context.json
is not a valid DFS filename.
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:195)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:104)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:396)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:392)
> 	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:392)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:336)
> 	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:908)
> 	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:889)
> 	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:786)
> 	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:775)
> 	at org.apache.falcon.workflow.WorkflowExecutionContext.serialize(WorkflowExecutionContext.java:291)
> 	at org.apache.falcon.workflow.WorkflowExecutionContext.serialize(WorkflowExecutionContext.java:276)
> 	at org.apache.falcon.workflow.FalconPostProcessing.run(FalconPostProcessing.java:47)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> 	at org.apache.falcon.workflow.FalconPostProcessing.main(FalconPostProcessing.java:37)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.oozie.action.hadoop.JavaMain.run(JavaMain.java:55)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message