hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
Date Fri, 15 Mar 2013 05:56:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603164#comment-13603164
] 

Bikas Saha commented on YARN-417:
---------------------------------

What is the deadlock here? Its late night and I cant see it :P
Is it related to the exception being thrown when stop() is called on the handler thread? Is
this guaranteed bad behavior and so we need to throw a runtime exception immediately?
I think we need to call client.stop() after the heartbeat thread has stopped. otherwise, the
heartbeat thread can call client.allocate() in between the current client.stop() and keepRunning=false,
right?
{code}
+  /**
+   * Tells the heartbeat and handler threads to stop and waits for them to
+   * terminate.  Calling this method from the callback handler thread would cause
+   * deadlock, and thus should be avoided.
+   */
+  @Override
+  public void stop() {
+    if (Thread.currentThread() == handlerThread) {
+      throw new YarnException("Cannot call stop from callback handler thread!");
+    }
+    client.stop();
+    keepRunning = false;
+    try {
+      heartbeatThread.join();
{code}

Didnt quite get the assert inside the loop. Perhaps you meant takeCompletedContainers()?
{code}
+    // wait for the allocated containers from the first heartbeat's response
+    while (callbackHandler.takeAllocatedContainers() == null) {
+      Assert.assertEquals(null, callbackHandler.takeAllocatedContainers());
+      Thread.sleep(10);
+    }
{code}

I think updating progress needs to be its own callback since its possible that no container
allocations and completions happen for a long time and thus the heartbeats show no progress
to the RM.
                
> Add a poller that allows the AM to receive notifications when it is assigned containers
> ---------------------------------------------------------------------------------------
>
>                 Key: YARN-417
>                 URL: https://issues.apache.org/jira/browse/YARN-417
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, applications
>    Affects Versions: 2.0.3-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>         Attachments: AMRMClientAsync-1.java, AMRMClientAsync.java, YARN-417-1.patch,
YARN-417-2.patch, YARN-417-3.patch, YARN-417-4.patch, YARN-417-4.patch, YARN-417.patch, YarnAppMaster.java,
YarnAppMasterListener.java
>
>
> Writing AMs would be easier for some if they did not have to handle heartbeating to the
RM on their own.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message