hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohith Sharma K S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3813) Support Application timeout feature in YARN.
Date Fri, 04 Sep 2015 05:39:47 GMT

    [ https://issues.apache.org/jira/browse/YARN-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730342#comment-14730342

Rohith Sharma K S commented on YARN-3813:

Thanks [~nijel] for the patch.. Some comments
# RMAppTimeOutMonitor should be part of RMActiveServiceContext. In RMContext, RMAppTimeOutMonitor
to RMActiveServiceContext.
# {{protected RMAppTimeOutMonitor rmAppTimeOutMonitor;}} in ResourceManager, can be moved
to RMActiveServices.
# RMAppImpl.java : applicationTimeout variable can be reused.
# RMAppTimeOutMonitor : {{private ConcurrentMap<RMApp, Long> rmApps =  new ConcurrentHashMap<RMApp,
Long>();}} , I think ApplicationId can be used. And for triggering KILL event, can get
RMApp from rmContext and trigger KILL event.
# RMAppTimeOutMonitor : When InterruptedException is thrown in the below code, thread should
 break or throw back exception. So, thread will die else thread wil be alive for ever
+        try {
+          Thread.sleep(threadSleepTime);
+        } catch (InterruptedException e1) {
+          LOG.debug("RMAppTimeOut sleep is over. Going for next iteration.");
+        }
# Tests : Need to add some more concrete tests covering up scenarios.

> Support Application timeout feature in YARN. 
> ---------------------------------------------
>                 Key: YARN-3813
>                 URL: https://issues.apache.org/jira/browse/YARN-3813
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: scheduler
>            Reporter: nijel
>            Assignee: nijel
>         Attachments: 0001-YARN-3813.patch, YARN Application Timeout .pdf
> It will be useful to support Application Timeout in YARN. Some use cases are not worried
about the output of the applications if the application is not completed in a specific time.

> *Background:*
> The requirement is to show the CDR statistics of last few  minutes, say for every 5 minutes.
The same Job will run continuously with different dataset.
> So one job will be started in every 5 minutes. The estimate time for this task is 2 minutes
or lesser time. 
> If the application is not completing in the given time the output is not useful.
> *Proposal*
> So idea is to support application timeout, with which timeout parameter is given while
submitting the job. 
> Here, user is expecting to finish (complete or kill) the application in the given time.
> One option for us is to move this logic to Application client (who submit the job). 
> But it will be nice if it can be generic logic and can make more robust.
> Kindly provide your suggestions/opinion on this feature. If it sounds good, i will update
the design doc and prototype patch

This message was sent by Atlassian JIRA

View raw message