hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
Date Tue, 19 Mar 2013 02:45:16 GMT

    [ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13605971#comment-13605971
] 

Bikas Saha commented on YARN-417:
---------------------------------

bq. It is guaranteed bad behavior - a thread that tries to join with itself will wait indefinitely.
Yes I see it now that my brain is less sleepy :P

bq. bq.I think we need to call client.stop() after the heartbeat thread has stopped.
bq. If unregisterApplicationMaster() is called before stop, this is handled by setting keepRunning
to false there and synchronizing on the AMRMClient in run().
I was actually talking about the allocate() call in the heartbeat thread which can happen
between client.stop() and heartbeathandler.join(). That allocate() call may create an exception
since the client could have already stopped on a different thread. If the heartbeat thread
is stopped first then this will not happen. 

bq.It just makes sure that the allocate callback isn't called twice, which I guess is a pretty
weak assert because there's no good reason that would happen. I can take it out if you think
it makes things less clear.
It would actually make the test flaky. I would change it to assert takeCompletedContainers()
is still null if the last takeAllocatedContainers() was null. The same check is performed
before the while loop.

bq.When AMRMClientAsync#setProgress is called, the updated progress will be transmitted with
the next heartbeat. This can be called whenever something occurs that advances the app's progress,
not just on callbacks. Would this not work with some ways of using this client that I am not
thinking of?
For good or bad, app progress is sent on every allocate call. Which means that the app must
keep setting a new value of progress on a periodic interval (that is related to the allocated
heartbeat interval). Given that some motivation of this async client is to hide this periodicity
from the app, I thought it made sense to getProgress() in a callback. The app could very well
update its internal progress value whenever it pleases but it wont have to think about updating
the RM periodically. The setProgress method would need to be removed. Does that clarify?

                
> Add a poller that allows the AM to receive notifications when it is assigned containers
> ---------------------------------------------------------------------------------------
>
>                 Key: YARN-417
>                 URL: https://issues.apache.org/jira/browse/YARN-417
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, applications
>    Affects Versions: 2.0.3-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>         Attachments: AMRMClientAsync-1.java, AMRMClientAsync.java, YARN-417-1.patch,
YARN-417-2.patch, YARN-417-3.patch, YARN-417-4.patch, YARN-417-4.patch, YARN-417.patch, YarnAppMaster.java,
YarnAppMasterListener.java
>
>
> Writing AMs would be easier for some if they did not have to handle heartbeating to the
RM on their own.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message