hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-674) Slow or failing DelegationToken renewals on submission itself make RM unavailable
Date Wed, 13 Nov 2013 17:51:26 GMT

    [ https://issues.apache.org/jira/browse/YARN-674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821586#comment-13821586

Vinod Kumar Vavilapalli commented on YARN-674:

More comments, hopefully the final round.
 - RMAppManager.submitApplication: Put a comment where you move apps to finish state saying
we are doing this before token-renewal so that we don't renew tokens for finished apps.
 - DelegationTokenRenewer
    -- isServiceStarted needs to be volatile?
    -- handleDTRenewerEvent -> handleDTRenewerAppSubmitEvent
    -- Add a comment in handleDTRenewerEvent to indicate why DTRenewer is starting the app
as opposed to RMAppManager.
    -- applicationFinished(DelegationTokenRenewerEvent evt) -> handleAppFinishEvent and
similarly addApplication(DelegationTokenRenewerAppSubmitEvent evt) -> handleAppsubmitEvent?
    -- Instead of putting renewerCount in the main code path, you can access the thread count
from ThreadPoolExecutor.getPoolSize() in the tests directly ?

 - DelegationTokenRenewerAppSubmitEvent can be nested class inside DelegationTokenRenewer?
This is not an event from outside the renewer. Similarly DelegationTokenRenewerEventType.
Either nest them in, or create a separate package.
 - testInvalidDelegationTokenApplicationSubmit, testInvalidDTWithAddApplication: Seem similar
but test different things. May be rename one or both?

> Slow or failing DelegationToken renewals on submission itself make RM unavailable
> ---------------------------------------------------------------------------------
>                 Key: YARN-674
>                 URL: https://issues.apache.org/jira/browse/YARN-674
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Omkar Vinit Joshi
>         Attachments: YARN-674.1.patch, YARN-674.2.patch, YARN-674.3.patch, YARN-674.4.patch,
YARN-674.5.patch, YARN-674.5.patch, YARN-674.6.patch
> This was caused by YARN-280. A slow or a down NameNode for will make it look like RM
is unavailable as it may run out of RPC handlers due to blocked client submissions.

This message was sent by Atlassian JIRA

View raw message