hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brook Zhou (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-7098) LocalizerRunner should immediately send heartbeat response LocalizerStatus.DIE when the Container transitions from LOCALIZING to KILLING
Date Mon, 28 Aug 2017 22:55:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-7098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Brook Zhou updated YARN-7098:
-----------------------------
    Description: 
Currently, the following can happen:

1. ContainerLocalizer heartbeats to ResourceLocalizationService.
2. LocalizerTracker.processHeartbeat verifies that there is a LocalizerRunner for the localizerId
(containerId). Starts executing 
3. Container receives kill event, goes from LOCALIZING -> KILLING. The LocalizerRunner
is removed from LocalizerTracker.
4. Since check (2) passed, LocalizerRunner sends heartbeat response with LocalizerStatus.LIVE
and the next file to download.

What should happen here is that (4) sends a LocalizerStatus.DIE, since (3) happened before
the heartbeat response in (4). This saves the container from potentially downloading an extra
resource which will end up being deleted anyway.

  was:
Currently, the following can happen:

1. ContainerLocalizer heartbeats to ResourceLocalizationService.
2. LocalizerTracker.processHeartbeat verifies that there is a LocalizerRunner for the localizerId
(containerId).
3. Container receives kill event, goes from LOCALIZING -> KILLING. The LocalizerRunner
is not removed from LocalizerTracker due to locking.
4. Since check (2) passed, LocalizerRunner sends heartbeat response with LocalizerStatus.LIVE
and the next file to download.

What should happen here is that (4) sends a LocalizerStatus.DIE, since (3) happened before
the heartbeat response in (4). This saves the container from potentially downloading an extra
resource which will end up being deleted anyway.


> LocalizerRunner should immediately send heartbeat response LocalizerStatus.DIE when the
Container transitions from LOCALIZING to KILLING
> ----------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-7098
>                 URL: https://issues.apache.org/jira/browse/YARN-7098
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Brook Zhou
>            Assignee: Brook Zhou
>            Priority: Minor
>
> Currently, the following can happen:
> 1. ContainerLocalizer heartbeats to ResourceLocalizationService.
> 2. LocalizerTracker.processHeartbeat verifies that there is a LocalizerRunner for the
localizerId (containerId). Starts executing 
> 3. Container receives kill event, goes from LOCALIZING -> KILLING. The LocalizerRunner
is removed from LocalizerTracker.
> 4. Since check (2) passed, LocalizerRunner sends heartbeat response with LocalizerStatus.LIVE
and the next file to download.
> What should happen here is that (4) sends a LocalizerStatus.DIE, since (3) happened before
the heartbeat response in (4). This saves the container from potentially downloading an extra
resource which will end up being deleted anyway.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message