hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4779) Fix AM container allocation logic in SLS
Date Mon, 09 May 2016 22:34:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-4779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15277209#comment-15277209
] 

Jian He commented on YARN-4779:
-------------------------------

- we can convert the appId to jobId and to a map lookup ?
{code}
for (AMSimulator ams : amMap.values()) {
{code}

- added variable not updated?
{code}
    int added = 0;
    for (ContainerSimulator cs : allMaps) {
      if (added >= mapTotal - mapFinished) {
        break;
      }
      pendingMaps.add(cs);
    }

    // And same, only add totalReduces - finishedReduces
    added = 0;
    for (ContainerSimulator cs : allReduces) {
      if (added >= reduceTotal - reduceFinished) {
        break;
      }
      pendingReduces.add(cs);
    }
{code}
- revert MRAMSimulator#sendContainerRequest method changes
- remove AMSimulator#getApplicationAttemptId 
- why call restart immediately after launch ?
{code}
    super.notifyAMContainerLaunched(masterContainer);
    restart();
{code}
- why 100000000L ? the underlying code use -1 for AM container
{code}
            se.getNmMap().get(amContainer.getNodeId())
                .addNewContainer(amContainer, 100000000L);
{code}

> Fix AM container allocation logic in SLS
> ----------------------------------------
>
>                 Key: YARN-4779
>                 URL: https://issues.apache.org/jira/browse/YARN-4779
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-4779.1.patch, YARN-4779.2.patch, YARN-4779.3.patch
>
>
> Currently, SLS uses unmanaged AM for simulated map-reduce applications. And first allocated
container for each app is considered to be the master container.
> This could be problematic when preemption happens. CapacityScheduler preempt AM container
at lowest priority, but the simulated AM container isn't recognized by scheduler -- it is
a normal container from scheduler's perspective.
> This JIRA tries to fix this logic: do the real AM allocation instead of using unmanaged
AM.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message