hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-1776) renewDelegationToken should survive RM failover
Date Thu, 20 Mar 2014 19:55:44 GMT

     [ https://issues.apache.org/jira/browse/YARN-1776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhijie Shen updated YARN-1776:
------------------------------

    Attachment: YARN-1776.2.patch

I created a new patch, though I cannot do much to make sure updating DT is completely atomic
(which touches more than one file) in FileSystemRMStateStore, I try to make sure update a
file behave somehow atomically. It is useful for update DT, App and AppAttempt.

To update a file:
1. write new file to .new
2. delete the current file
3. rename .new to the current file

Here, write is FileSystemRMStateStore#writeFile to ensure atomic. Then, if RM fails before
1, nothing needs to be changed. If RM fails between 1 and 2, and between 2 and 3, we have
the .new in hand, we can replace the current file with it when loading the state information.
If RM fails after 3, nothing needs to be changed as well.

Please let me know if you want to combine these fix together, or split it into two tickets.

> renewDelegationToken should survive RM failover
> -----------------------------------------------
>
>                 Key: YARN-1776
>                 URL: https://issues.apache.org/jira/browse/YARN-1776
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
>         Attachments: YARN-1776.1.patch, YARN-1776.2.patch
>
>
> When a delegation token is renewed, two RMStateStore operations: 1) removing the old
DT, and 2) storing the new DT will happen. If RM fails in between. There would be problem.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message