hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
Date Fri, 15 Mar 2013 05:56:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603164#comment-13603164

Bikas Saha commented on YARN-417:

What is the deadlock here? Its late night and I cant see it :P
Is it related to the exception being thrown when stop() is called on the handler thread? Is
this guaranteed bad behavior and so we need to throw a runtime exception immediately?
I think we need to call client.stop() after the heartbeat thread has stopped. otherwise, the
heartbeat thread can call client.allocate() in between the current client.stop() and keepRunning=false,
+  /**
+   * Tells the heartbeat and handler threads to stop and waits for them to
+   * terminate.  Calling this method from the callback handler thread would cause
+   * deadlock, and thus should be avoided.
+   */
+  @Override
+  public void stop() {
+    if (Thread.currentThread() == handlerThread) {
+      throw new YarnException("Cannot call stop from callback handler thread!");
+    }
+    client.stop();
+    keepRunning = false;
+    try {
+      heartbeatThread.join();

Didnt quite get the assert inside the loop. Perhaps you meant takeCompletedContainers()?
+    // wait for the allocated containers from the first heartbeat's response
+    while (callbackHandler.takeAllocatedContainers() == null) {
+      Assert.assertEquals(null, callbackHandler.takeAllocatedContainers());
+      Thread.sleep(10);
+    }

I think updating progress needs to be its own callback since its possible that no container
allocations and completions happen for a long time and thus the heartbeats show no progress
to the RM.
> Add a poller that allows the AM to receive notifications when it is assigned containers
> ---------------------------------------------------------------------------------------
>                 Key: YARN-417
>                 URL: https://issues.apache.org/jira/browse/YARN-417
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, applications
>    Affects Versions: 2.0.3-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>         Attachments: AMRMClientAsync-1.java, AMRMClientAsync.java, YARN-417-1.patch,
YARN-417-2.patch, YARN-417-3.patch, YARN-417-4.patch, YARN-417-4.patch, YARN-417.patch, YarnAppMaster.java,
> Writing AMs would be easier for some if they did not have to handle heartbeating to the
RM on their own.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message