hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2052) ContainerId creation after work preserving restart is broken
Date Thu, 19 Jun 2014 20:59:24 GMT

    [ https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037875#comment-14037875
] 

Jian He commented on YARN-2052:
-------------------------------

I think the conclusion was to not add any new fields into ContainerId. Instead, we persist
the epoch number. Each time restart happens, the initial value of AppSchedulingInfo#containerIdCounter
will increase by (epoch*2^22) if we reserve 10bits for the number of RM restarts.  Later on
if we change the int to long, we will have 2^32 for epoch number which should be fairly enough.
This patch should include state-store change as well as the containerIdCounter change.

> ContainerId creation after work preserving restart is broken
> ------------------------------------------------------------
>
>                 Key: YARN-2052
>                 URL: https://issues.apache.org/jira/browse/YARN-2052
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Tsuyoshi OZAWA
>            Assignee: Tsuyoshi OZAWA
>         Attachments: YARN-2052.1.patch, YARN-2052.2.patch, YARN-2052.3.patch, YARN-2052.4.patch
>
>
> Container ids are made unique by using the app identifier and appending a monotonically
increasing sequence number to it. Since container creation is a high churn activity the RM
does not store the sequence number per app. So after restart it does not know what the new
sequence number should be for new allocations.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message