hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1954) Add waitFor to AMRMClient(Async)
Date Mon, 21 Jul 2014 11:11:39 GMT

    [ https://issues.apache.org/jira/browse/YARN-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068411#comment-14068411

Zhijie Shen commented on YARN-1954:

[~ozawa], I'm generally fine with the patch. Just some minor comments.

1. Throw IllegalArgumentException instead?
+    if (checkEveryMillis <= 0) {
+      checkEveryMillis = 10000;
+    }

2. What if checkEveryMillis > 60000? Maybe we can simply hard code the fix number of rounds
to output a warning log. And don't output a warning long in each round, and a info log at
regular intervals. How do you think?
+    final int loggingCounterInitValue = 60000 / checkEveryMillis;
+    int loggingCounter = loggingCounterInitValue;

3. There's unnecessary changes in AMRMClientAsyncImpl

4. In TestAMRMClient #testWaitFor, can you justify countDownChecker#counter == 3 after waitFor.

5. Not necessary to be in a synchronized block if you don't use wait here.
+    synchronized (callbackHandler.notifier) {
+      asyncClient.registerApplicationMaster("localhost", 1234, null);
+      asyncClient.waitFor(checker);
+    }

> Add waitFor to AMRMClient(Async)
> --------------------------------
>                 Key: YARN-1954
>                 URL: https://issues.apache.org/jira/browse/YARN-1954
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: client
>    Affects Versions: 3.0.0, 2.4.0
>            Reporter: Zhijie Shen
>            Assignee: Tsuyoshi OZAWA
>         Attachments: YARN-1954.1.patch, YARN-1954.2.patch, YARN-1954.3.patch, YARN-1954.4.patch,
> Recently, I saw some use cases of AMRMClient(Async). The painful thing is that the main
non-daemon thread has to sit in a dummy loop to prevent AM process exiting before all the
tasks are done, while unregistration is triggered on a separate another daemon thread by callback
methods (in particular when using AMRMClientAsync). IMHO, it should be beneficial to add a
waitFor method to AMRMClient(Async) to block the AM until unregistration or user supplied
check point, such that users don't need to write the loop themselves.

This message was sent by Atlassian JIRA

View raw message