falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Srikanth Sundarrajan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-991) consolidate get coord actions in OozieWorkflowEngine
Date Thu, 04 Jun 2015 04:08:39 GMT

    [ https://issues.apache.org/jira/browse/FALCON-991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14572084#comment-14572084
] 

Srikanth Sundarrajan commented on FALCON-991:
---------------------------------------------

[~ajayyadava]
{code}
+                    coordJob = client.getCoordJobInfo(coord.getId(), "nominaltime>=" +
start + ";nominaltime<" + end,
+                            0, Integer.MAX_VALUE);
{code}

Would retrieving all the coord actions for a coord cause OOM within Falcon with this change
? From the looks of it, this wont be any worser than what it was before this. Please confirm

Also can the exception be added to LOG statement ?

{code}
+                    LOG.error("Unable to getcoordinator job for:{} for start:{} and end:
{}", coord.getId(),
+                            start, end);
{code}

should perhaps be

{code}
+                    LOG.error("Unable to getcoordinator job for:{} for start:{} and end:
{}", coord.getId(),
+                            start, end, e);
{code}

> consolidate get coord actions in OozieWorkflowEngine
> ----------------------------------------------------
>
>                 Key: FALCON-991
>                 URL: https://issues.apache.org/jira/browse/FALCON-991
>             Project: Falcon
>          Issue Type: Improvement
>            Reporter: Shwetha G S
>            Assignee: Ajay Yadava
>         Attachments: FALCON-991-v1.patch, FALCON-991-v2.patch, FALCON-991.patch
>
>
> OozieWorkflowEngine.getCoordActions() gets instances by calling oozieclient.getCoordAction()
for each action. Its not efficient to call oozie API for each instance. Instead, doSummaryJobAction()
gets all instances in bulk using oozieclient.getCoordJobInfo(). But this uses offset which
won't work with purging of coord actions. A new API was added in oozie to get coord actions
between timestamps. We should use that for both usecases



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message