hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billie Rinaldi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8710) Service AM should set a finite limit on NM container max retries
Date Fri, 05 Oct 2018 21:22:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16640361#comment-16640361

Billie Rinaldi commented on YARN-8710:

I think 3 retries are a bit low over an infinite failure validity interval. Maybe 6 or 10
retries (with 1 retry every 30 seconds, that's 3 or 5 minutes of retrying) over a failure
validity interval of 10 or 15 minutes would be appropriate. Components can configure different
values if necessary.

The total retry count is controlled by yarn.service.container-failure-per-component.threshold.
It counts container failures per component and does not count NM container retries (the AM
is not currently informed about retries, see YARN-8286). The default value is 10 failures
per component over a 6 hour window.

> Service AM should set a finite limit on NM container max retries 
> -----------------------------------------------------------------
>                 Key: YARN-8710
>                 URL: https://issues.apache.org/jira/browse/YARN-8710
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn-native-services
>            Reporter: Suma Shivaprasad
>            Assignee: Suma Shivaprasad
>            Priority: Major
>         Attachments: YARN-8710.1.patch
> Container retries are currently set to a default of -1 in AbstractProviderService.buildContainerRetry.
If this is not overridden via service spec with a finite value for yarn.service.container-failure.retry.max
, this causes infinite NM reties for the container for ALWAYS/ON_FAILURE restart policy .
Ideally it should try a finite number of time on the same NM and subsequently Service AM can
retry on another node.
> We can set this to default value of 3.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message