Mailing-List: contact yarn-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of
 prvs=1979c8f3a=criccomini@linkedin.com designates 69.28.149.80 as permitted
 sender)
From: Chris Riccomini <criccomini@linkedin.com>
To: "yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>
Subject: Host-specific ContainerRequest ignored
Thread-Topic: Host-specific ContainerRequest ignored
Thread-Index: AQHPaiy61nPXPW8vjESusNToeYFwiw==
Date: Wed, 7 May 2014 19:44:16 +0000
Message-ID: <CF8FDB1C.287B9%criccomini@linkedin.com>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/14.4.1.140326
Content-Type: multipart/alternative;
	boundary="_000_CF8FDB1C287B9criccominilinkedincom_"
MIME-Version: 1.0

--_000_CF8FDB1C287B9criccominilinkedincom_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hey Guys,

I am creating a container request:


  protected def requestContainers(memMb: Int, cpuCores: Int, containers: In=
t) {

    info("Requesting %d container(s) with %dmb of memory" format (container=
s, memMb))

    val capability =3D Records.newRecord(classOf[Resource])

    val priority =3D Records.newRecord(classOf[Priority])

    priority.setPriority(0)

    capability.setMemory(memMb)

    capability.setVirtualCores(cpuCores)

    (0 until containers).foreach(idx =3D> amClient.addContainerRequest(new =
ContainerRequest(capability, null, null, priority)))

  }

This pretty closely mirrors the distributed shell example.

If I put an array with a host string in the ContainerRequest, YARN seems to=
 completely ignore this request, and continues to put all containers on one=
 or two nodes in the grid, which aren't the ones I requested, even though t=
he grid is completely empty, and there are 15 nodes available. This also ho=
lds true if I put "false" for relax locality. I'm running the CapacitySched=
uler with a node-locality-delay set to 40. Previously, I tried the FifoSche=
duler, and it exhibited the same behavior.

All NMs are just using the /default-rack for their rack. The strings that I=
'm putting in the hosts String[] parameter in ContainerRequest are hard cod=
ed to exactly match the NodeIds being listed in the NMs.

What am I doing wrong? I feel like I'm missing some configuration on the ca=
pacity scheduler or NMs or something.

Cheers,
Chris

--_000_CF8FDB1C287B9criccominilinkedincom_--