falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Yadava (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-1196) Partial submission of cluster.xml on Prism
Date Wed, 06 May 2015 03:47:59 GMT

    [ https://issues.apache.org/jira/browse/FALCON-1196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529845#comment-14529845

Ajay Yadava commented on FALCON-1196:


You have guessed it right. This is an architectural decision to enforce idempotency. Same
behavior exists for processes and feeds as well. 
In more plain terms, for this case, you can submit the same cluster multiple times without
any undesired side effects.

It simplifies lot of things in falcon. One of the scenarios where this behavior is useful
is as follows. Assume that you have to submit a process which runs on several clusters. If
few of the clusters have an issue because of which it couldn't be submitted (say a missing
feed dependency) then you can fix that issue and then submit the same process on the prism
again. It will submit the same process again on all falcon servers (even the ones where it
was earlier submitted successfully). It will not have any side effect on the servers where
it was submitted successfully earlier and it will be submitted successfully on the servers
where it had failed earlier. Prism doesn't need to maintain state or query where it was successful
earlier and where it had failed. Also, you don't have to go to each cluster where it failed
earlier and submit the process again, you can just submit it again at the prism in one step.

That said, the message can be improved in certain cases and there is already a JIRA for one
of those cases(FALCON-75). You can find more on idempotency in FALCON-75 & http://falcon.apache.org/0.6-incubating/FalconDocumentation.html#Idempotency

Hope it helps!

> Partial submission of cluster.xml on Prism
> ------------------------------------------
>                 Key: FALCON-1196
>                 URL: https://issues.apache.org/jira/browse/FALCON-1196
>             Project: Falcon
>          Issue Type: Bug
>            Reporter: Mahak
> When Prism fails to submit a cluster on some falcon instances, the changes on the successful
servers aren't rolled back. So if one queries any of those successful servers locally the
clusters show up in the list. 
> This behavior is irrespective of if the falcon instance in the colo, the cluster.xml
is written for, was successful or not.
> This doesn't really throw any exceptions/errors on later submission of the same cluster.
However, may result in logical fallacies if one were to locally query such a partially updated
> I'm not sure if the same behavior is exhibited when submitting feeds/processes. 

This message was sent by Atlassian JIRA

View raw message