falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Yadava (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1401) MetadataMappingService fails to add an edge for a process instance
Date Mon, 14 Sep 2015 10:12:45 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14743320#comment-14743320
] 

Ajay Yadava commented on FALCON-1401:
-------------------------------------

The "magical" value (IGNORE) is added when outputs don't exist, so what does it mean that
there is an output feed? 

Moreover, whatever be the thinking behind it (changing from "NONE" to "IGNORE"), it is wrong
to support two magical values, even worse if you can't make out one value by reading the code.
It is for all present and future consumers of those arguments and shouldn't be changed.

We shouldn't change magical values conditionally also, if there are use cases to provide extra
signals then we should add a new arg. If a feed exists then it's name should always be present,
it should be left to consumer to decide what to ignore, as contrary to one consumer other
consumers might choose to not ignore it. Ideally it should have been caught in review of FALCON-1091
but even now we should revert to a single value of "NONE".




> MetadataMappingService fails to add an edge for a process instance
> ------------------------------------------------------------------
>
>                 Key: FALCON-1401
>                 URL: https://issues.apache.org/jira/browse/FALCON-1401
>             Project: Falcon
>          Issue Type: Bug
>            Reporter: Pallavi Rao
>            Assignee: Pallavi Rao
>            Priority: Minor
>         Attachments: FALCON-1401.patch
>
>
> If a process has no output feeds, the metadata mapping service fails to add an edge for
the process instances. It fails with NPE:
> {code}
> 2015-08-14 15:31:00,950 INFO  - [ActiveMQ Session Task:] ~ Adding process instance: pig-sleep-process/2015-07-28T07:40Z
(InstanceRelationshipGraphBuilder:74)
> 2015-08-14 15:31:00,951 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex for: name=pig-sleep-process/2015-07-28T07:40Z,
type=PROCESS_INSTANCE (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:00,957 DEBUG - [ActiveMQ Session Task:] ~ Creating a new vertex for:
name=pig-sleep-process/2015-07-28T07:40Z, type=PROCESS_INSTANCE (RelationshipGraphBuilder:101)
> 2015-08-14 15:31:00,981 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex for: name=pig-sleep-process,
type=PROCESS_ENTITY (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:00,984 INFO  - [ActiveMQ Session Task:] ~ Vertex exists? name=pig-sleep-process,
type=PROCESS_ENTITY, v=v[16] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:00,988 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex for: name=local,
type=CLUSTER_ENTITY (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:00,990 INFO  - [ActiveMQ Session Task:] ~ Vertex exists? name=local,
type=CLUSTER_ENTITY, v=v[4] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:00,992 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex for: name=pallavi.rao,
type=USER (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:00,995 INFO  - [ActiveMQ Session Task:] ~ Vertex exists? name=pallavi.rao,
type=USER, v=v[8] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:00,997 INFO  - [ActiveMQ Session Task:] ~ Computing feed instance for
: name= IGNORE path= IGNORE, in cluster: local (InstanceRelationshipGraphBuilder:243)
> 2015-08-14 15:31:01,009 INFO  - [ActiveMQ Session Task:] ~ Adding process instance: pig-sleep-process/2015-07-28T07:40Z
(InstanceRelationshipGraphBuilder:74)
> 2015-08-14 15:31:01,009 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex for: name=pig-sleep-process/2015-07-28T07:40Z,
type=PROCESS_INSTANCE (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,012 DEBUG - [ActiveMQ Session Task:] ~ Creating a new vertex for:
name=pig-sleep-process/2015-07-28T07:40Z, type=PROCESS_INSTANCE (RelationshipGraphBuilder:101)
> 2015-08-14 15:31:01,033 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex for: name=pig-sleep-process,
type=PROCESS_ENTITY (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,035 INFO  - [ActiveMQ Session Task:] ~ Vertex exists? name=pig-sleep-process,
type=PROCESS_ENTITY, v=v[16] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:01,039 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex for: name=local,
type=CLUSTER_ENTITY (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,041 INFO  - [ActiveMQ Session Task:] ~ Vertex exists? name=local,
type=CLUSTER_ENTITY, v=v[4] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:01,043 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex for: name=pallavi.rao,
type=USER (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,044 INFO  - [ActiveMQ Session Task:] ~ Vertex exists? name=pallavi.rao,
type=USER, v=v[8] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:01,047 INFO  - [ActiveMQ Session Task:] ~ Computing feed instance for
: name= IGNORE path= IGNORE, in cluster: local (InstanceRelationshipGraphBuilder:243)
> 2015-08-14 15:31:01,073 INFO  - [ActiveMQ Session Task:] ~ Adding process instance: pig-sleep-process/2015-07-28T07:40Z
(InstanceRelationshipGraphBuilder:74)
> 2015-08-14 15:31:01,073 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex for: name=pig-sleep-process/2015-07-28T07:40Z,
type=PROCESS_INSTANCE (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,076 DEBUG - [ActiveMQ Session Task:] ~ Creating a new vertex for:
name=pig-sleep-process/2015-07-28T07:40Z, type=PROCESS_INSTANCE (RelationshipGraphBuilder:101)
> 2015-08-14 15:31:01,096 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex for: name=pig-sleep-process,
type=PROCESS_ENTITY (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,098 INFO  - [ActiveMQ Session Task:] ~ Vertex exists? name=pig-sleep-process,
type=PROCESS_ENTITY, v=v[16] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:01,102 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex for: name=local,
type=CLUSTER_ENTITY (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,103 INFO  - [ActiveMQ Session Task:] ~ Vertex exists? name=local,
type=CLUSTER_ENTITY, v=v[4] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:01,105 DEBUG - [ActiveMQ Session Task:] ~ Finding vertex for: name=pallavi.rao,
type=USER (RelationshipGraphBuilder:87)
> 2015-08-14 15:31:01,108 INFO  - [ActiveMQ Session Task:] ~ Vertex exists? name=pallavi.rao,
type=USER, v=v[8] (InstanceRelationshipGraphBuilder:129)
> 2015-08-14 15:31:01,110 INFO  - [ActiveMQ Session Task:] ~ Computing feed instance for
: name= IGNORE path= IGNORE, in cluster: local (InstanceRelationshipGraphBuilder:243)
> 2015-08-14 15:31:01,111 ERROR - [ActiveMQ Session Task:] ~ Error in listener org.apache.falcon.metadata.MetadataMappingService
(WorkflowJobEndNotificationService:226)
> org.apache.falcon.FalconException: java.lang.RuntimeException: java.lang.NullPointerException
> 	at org.apache.falcon.metadata.MetadataMappingService.onSuccess(MetadataMappingService.java:274)
> 	at org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifyWorkflowEnd(WorkflowJobEndNotificationService.java:222)
> 	at org.apache.falcon.workflow.WorkflowJobEndNotificationService.notifySuccess(WorkflowJobEndNotificationService.java:105)
> 	at org.apache.falcon.messaging.JMSMessageConsumer.invokeListener(JMSMessageConsumer.java:169)
> 	at org.apache.falcon.messaging.JMSMessageConsumer.onMessage(JMSMessageConsumer.java:112)
> 	at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1229)
> 	at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:134)
> 	at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:205)
> 	at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
> 	at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException
> 	at com.tinkerpop.blueprints.util.TransactionRetryStrategy$AbstractRetryStrategy.execute(TransactionRetryStrategy.java:176)
> 	at com.tinkerpop.blueprints.util.TransactionRetryHelper.use(TransactionRetryHelper.java:115)
> 	at com.tinkerpop.blueprints.util.TransactionRetryHelper.exponentialBackoff(TransactionRetryHelper.java:94)
> 	at org.apache.falcon.metadata.MetadataMappingService.onSuccess(MetadataMappingService.java:263)
> 	... 12 more
> Caused by: java.lang.NullPointerException
> 	at org.apache.falcon.entity.FeedHelper.getCluster(FeedHelper.java:80)
> 	at org.apache.falcon.entity.FeedHelper.getStorageType(FeedHelper.java:235)
> 	at org.apache.falcon.metadata.InstanceRelationshipGraphBuilder.getFeedInstanceName(InstanceRelationshipGraphBuilder.java:280)
> 	at org.apache.falcon.metadata.InstanceRelationshipGraphBuilder.addFeedInstance(InstanceRelationshipGraphBuilder.java:245)
> 	at org.apache.falcon.metadata.InstanceRelationshipGraphBuilder.addOutputFeedInstances(InstanceRelationshipGraphBuilder.java:151)
> 	at org.apache.falcon.metadata.MetadataMappingService.onProcessInstanceExecuted(MetadataMappingService.java:313)
> 	at org.apache.falcon.metadata.MetadataMappingService.onSuccessfulExecution(MetadataMappingService.java:282)
> 	at org.apache.falcon.metadata.MetadataMappingService.access$100(MetadataMappingService.java:53)
> 	at org.apache.falcon.metadata.MetadataMappingService$3.execute(MetadataMappingService.java:267)
> 	at org.apache.falcon.metadata.MetadataMappingService$3.execute(MetadataMappingService.java:264)
> 	at com.tinkerpop.blueprints.util.TransactionRetryStrategy$AbstractRetryStrategy.execute(TransactionRetryStrategy.java:146)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message