hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-417) Add a poller that allows the AM to receive notifications when it is assigned containers
Date Tue, 26 Feb 2013 17:56:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13587312#comment-13587312
] 

Bikas Saha commented on YARN-417:
---------------------------------

Yes. add/release should be used instead of allocate because allocate makes an RPC that is
subject to failures and race conditions. Its all handled in AMRMClient.

While I like onContainer*() we need to think about proliferation of such methods. A huge method
listing tends to make me cautious because I have to understand all those methods. Also, I
have a gut feeling that there might be a lot of user code replication for each callback. I
like it when code is in one place. How about an onContainer"GoodName"() and then provide a
helper method that tells whether the container is completed, successfull, killed, outofmemory.
IMO, this might be more flexible and less verbose. And the user handling logic is in one place.

One other thought, is that where possible, the callbacks should return collection/list of
objects. Say, we have a large job and we are getting 100's of containers. For perf, it might
be better to not have 100's of calls for onContainerAllocated() and onContainerCompleted().

Now that we have 2 example codes and some discussion, lets post a concrete interface declaration
with javadoc. That will help focus the discussion and we can converge quickly.
                
> Add a poller that allows the AM to receive notifications when it is assigned containers
> ---------------------------------------------------------------------------------------
>
>                 Key: YARN-417
>                 URL: https://issues.apache.org/jira/browse/YARN-417
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, applications
>    Affects Versions: 2.0.3-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>         Attachments: YARN-417.patch, YarnAppMaster.java, YarnAppMasterListener.java
>
>
> Writing AMs would be easier for some if they did not have to handle heartbeating to the
RM on their own.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message