hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
Date Tue, 26 Feb 2013 03:48:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586702#comment-13586702

Sandy Ryza commented on YARN-417:

Thanks for the comments Bikas, and for your perspective, Chris.


Your code brings up that we should probably have a reboot callback as well.  I also like the
idea of onContainerLost/onContainerFail/onContainerSuccess.  Do you have an opinion on whether
more of the lifecycle should be handled here?  My initial inclination would be to do that
separately/on top of the async client.

To reduce the number of classes a user needs to deal with, we could have Heartbeater wrap
the AMRMClient methods and rename it something like AsyncAMRMClient. To also eliminate the
AllocationListener interface, we could have the user subclass Heartbeater?  For the latter,
though, in this case to me registering callbacks seems like an easier mental model than subclassing,
as it better separates the client from the user code in the user's mind.  But I don't feel
super strongly about this.

The futures idea is interesting - if submitContainerRequest returned a future it would be
cool that someone could scheduler containers just using:
List<Container> myAllocatedContainers = heartbeater.submitContainerRequests().getContainers();
However, I don't think this captures enough of a majority of use cases, as many YARN apps
will want to start containers asynchronously as they trickle back in.

Regarding the callbacks, I didn't notice that the info returned in AMResponse is stored in
AMRMClient.  As the user can get access to it that way, I agree, Bikas, it's probably clearer
to take out the allocateResponse method.  Regarding the completedContainers and allocatedContainers
as two methods or one, I don't feel super strongly about either approach, but if all else
is equal, to me it seems easier for the developer to think about them separately, especially
with reboot and if we want to do the onContainerLost/onContainerFail/onContainerSuccess that
Chris mentioned.  In java awt/swing, mouseReleased and mouseClicked events come at the same
time, rather than mouseReleased containing an isClick param, and this always seemed intuitive
to me.  I can see that it might be useful to know what containers were finished before knowing
what containers were allocated, but I can't come up with a reason for the other way around.

> Add a poller that allows the AM to receive notifications when it is assigned containers
> ---------------------------------------------------------------------------------------
>                 Key: YARN-417
>                 URL: https://issues.apache.org/jira/browse/YARN-417
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, applications
>    Affects Versions: 2.0.3-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>         Attachments: YARN-417.patch, YarnAppMaster.java, YarnAppMasterListener.java
> Writing AMs would be easier for some if they did not have to handle heartbeating to the
RM on their own.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message