hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2313) Livelock can occur in FairScheduler when there are lots of running apps
Date Wed, 23 Jul 2014 09:27:39 GMT

    [ https://issues.apache.org/jira/browse/YARN-2313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14071547#comment-14071547
] 

Karthik Kambatla commented on YARN-2313:
----------------------------------------

Sorry for coming in late here. Didn't see this before.

I think we need a better solution here. Otherwise, clusters will continue to run into this.


One simple way to address this could be to wait {{updateInterval}} ms after finishing an iteration
of update-thread before starting the next iteration. We should do something similar for the
continuous thread as well. 

> Livelock can occur in FairScheduler when there are lots of running apps
> -----------------------------------------------------------------------
>
>                 Key: YARN-2313
>                 URL: https://issues.apache.org/jira/browse/YARN-2313
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 2.4.1
>            Reporter: Tsuyoshi OZAWA
>            Assignee: Tsuyoshi OZAWA
>             Fix For: 2.6.0
>
>         Attachments: YARN-2313.1.patch, YARN-2313.2.patch, YARN-2313.3.patch, YARN-2313.4.patch,
rm-stack-trace.txt
>
>
> Observed livelock on FairScheduler when there are lots entry in queue. After my investigating
code, following case can occur:
> 1. {{update()}} called by UpdateThread takes longer times than UPDATE_INTERVAL(500ms)
if there are lots queue.
> 2. UpdateThread goes busy loop.
> 3. Other threads(AllocationFileReloader, ResourceManager$SchedulerEventDispatcher) can
wait forever.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message