hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsuyoshi OZAWA (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2229) ContainerId can overflow with RM restart
Date Wed, 10 Sep 2014 23:07:35 GMT

     [ https://issues.apache.org/jira/browse/YARN-2229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tsuyoshi OZAWA updated YARN-2229:
---------------------------------
    Attachment: YARN-2229.15.patch

Talked with Jian offline.  Updated to reduce epoch bits from 32 bits to 24 bits and increase
id bits from 32 bits to 40 bits because 32 bits for epoch is too much. It is allowed to truncate
int32/64 values by the spec of protobuf.
https://developers.google.com/protocol-buffers/docs/proto


> ContainerId can overflow with RM restart
> ----------------------------------------
>
>                 Key: YARN-2229
>                 URL: https://issues.apache.org/jira/browse/YARN-2229
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Tsuyoshi OZAWA
>            Assignee: Tsuyoshi OZAWA
>         Attachments: YARN-2229.1.patch, YARN-2229.10.patch, YARN-2229.10.patch, YARN-2229.11.patch,
YARN-2229.12.patch, YARN-2229.13.patch, YARN-2229.14.patch, YARN-2229.15.patch, YARN-2229.2.patch,
YARN-2229.2.patch, YARN-2229.3.patch, YARN-2229.4.patch, YARN-2229.5.patch, YARN-2229.6.patch,
YARN-2229.7.patch, YARN-2229.8.patch, YARN-2229.9.patch
>
>
> On YARN-2052, we changed containerId format: upper 10 bits are for epoch, lower 22 bits
are for sequence number of Ids. This is for preserving semantics of {{ContainerId#getId()}},
{{ContainerId#toString()}}, {{ContainerId#compareTo()}}, {{ContainerId#equals}}, and {{ConverterUtils#toContainerId}}.
One concern is epoch can overflow after RM restarts 1024 times.
> To avoid the problem, its better to make containerId long. We need to define the new
format of container Id with preserving backward compatibility on this JIRA.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message