hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsuyoshi OZAWA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1954) Add waitFor to AMRMClient(Async)
Date Fri, 01 Aug 2014 21:47:40 GMT

    [ https://issues.apache.org/jira/browse/YARN-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14083011#comment-14083011

Tsuyoshi OZAWA commented on YARN-1954:

Thanks you for the review, [~zjshen]. I'm creating a patch to address your comment.

2. What if checkEveryMillis > 60000? Maybe we can simply hard code the fix number of rounds
to output a warning log. And don't output a warning long in each round, and a info log at
regular intervals. How do you think?

If checkEveryMillis > 60000, it will log every time after thread's waking up. It's same
to loggingCounterInitValue == 1. You may concern that it will log the info too much, but checkEveryMillis
is larger than 60000. Therefore, I think it's no problem. What do you think this behavior?

> Add waitFor to AMRMClient(Async)
> --------------------------------
>                 Key: YARN-1954
>                 URL: https://issues.apache.org/jira/browse/YARN-1954
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: client
>    Affects Versions: 3.0.0, 2.4.0
>            Reporter: Zhijie Shen
>            Assignee: Tsuyoshi OZAWA
>         Attachments: YARN-1954.1.patch, YARN-1954.2.patch, YARN-1954.3.patch, YARN-1954.4.patch,
> Recently, I saw some use cases of AMRMClient(Async). The painful thing is that the main
non-daemon thread has to sit in a dummy loop to prevent AM process exiting before all the
tasks are done, while unregistration is triggered on a separate another daemon thread by callback
methods (in particular when using AMRMClientAsync). IMHO, it should be beneficial to add a
waitFor method to AMRMClient(Async) to block the AM until unregistration or user supplied
check point, such that users don't need to write the loop themselves.

This message was sent by Atlassian JIRA

View raw message