hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei Yan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3020) n similar addContainerRequest()s produce n*(n+1)/2 containers
Date Fri, 09 Jan 2015 16:42:35 GMT

    [ https://issues.apache.org/jira/browse/YARN-3020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271504#comment-14271504
] 

Wei Yan commented on YARN-3020:
-------------------------------

[~peterdkirchner], I run the following test code in TestAMRMClient.java, and it passed. This
testcase lets amClient call addContainerRequest() 10 times with the same priority, and the
result is 10 requests. Did I misunderstand your point?
{code}
  @Test
  public void tempAddContainerRequest() throws YarnException, IOException {
    AMRMClient<ContainerRequest> amClient = null;
    try {
      // start am rm client
      amClient = AMRMClient.<ContainerRequest>createAMRMClient();
      amClient.init(conf);
      amClient.start();
      amClient.registerApplicationMaster("Host", 10000, "");

      Resource capacity = Resource.newInstance(1024, 2);
      
      // Same priority
      for (int i = 0; i < 10; i ++) {
        ContainerRequest request = new ContainerRequest(capacity, null, null, priority);
        amClient.addContainerRequest(request);
      }
      List list1 = amClient.getMatchingRequests(priority, "*", capacity);
      assertEquals(1, list1.size());
      LinkedHashSet set1 = (java.util.LinkedHashSet)(list1.get(0));
      assertEquals(10, set1.size());

      amClient.unregisterApplicationMaster(FinalApplicationStatus.SUCCEEDED,
          null, null);
    } finally {
      if (amClient != null && amClient.getServiceState() == STATE.STARTED) {
        amClient.stop();
      }
    }
  }
{code}

> n similar addContainerRequest()s produce n*(n+1)/2 containers
> -------------------------------------------------------------
>
>                 Key: YARN-3020
>                 URL: https://issues.apache.org/jira/browse/YARN-3020
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 2.5.0, 2.6.0, 2.5.1, 2.5.2
>            Reporter: Peter D Kirchner
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> BUG: If the application master calls addContainerRequest() n times, but with the same
priority, I get 1+2+3+...+n containers = n*(n+1)/2 .
> If the application master calls addContainerRequest() n times, but with a unique priority
each time, I get n containers (as I intended).
> Analysis:
> There is a logic problem in AMRMClientImpl.java.
> Although AMRMClientImpl.java, allocate() does an ask.clear() , on subsequent calls to
addContainerRequest(), addResourceRequest() finds the previous matching remoteRequest and
increments the container count rather than starting anew, and does an addResourceRequestToAsk()
which defeats the ask.clear().
> From documentation and code comments, it was hard for me to discern the intended behavior
of the API, but the inconsistency reported in this issue suggests one case or the other is
implemented incorrectly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message