reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dhruv Mahajan (JIRA)" <>
Subject [jira] [Created] (REEF-1653) Request for evaluator with multiple cores always return evaluator with 1 core only in IMRU
Date Tue, 25 Oct 2016 00:26:58 GMT
Dhruv Mahajan created REEF-1653:

             Summary: Request for evaluator with multiple cores always return evaluator with
1 core only in IMRU
                 Key: REEF-1653
             Project: REEF
          Issue Type: Bug
          Components: REEF.NET
         Environment: C#
            Reporter: Dhruv Mahajan

Currently, on submitting request for evaluators with multiple cores in IMRU to YARN cluster,
we always get back evaluators with 1 core only. Memory allocated is right and higher rounded
off value than requested which is the desired behavior. However, number of cores has issues.
Find below relevant logs.

I asked for evaluator with 7000MB memory and 7 cores. In return I got one with around 9000MB
memory and 1 core only which does not seem good right? Find below relevant logs from driver:

TLCPP, Version=, Culture=neutral, PublicKeyToken=null],[TLCPP.LBFGS.MapFunctionInputOutput.MapFunctionOutput,
TLCPP, Version=, Culture=neutral, PublicKeyToken=null],[Microsoft.MachineLearning.Data.VBuffer`1[[System.Single,
mscorlib, Version=, Culture=neutral, PublicKeyToken=b77a5c561934e089]], Microsoft.MachineLearning.Core,
Version=, Culture=neutral, PublicKeyToken=d353f9ba84f0e281],[Microsoft.MachineLearning.Data.RoleMappedData,
Microsoft.MachineLearning.Core, Version=, Culture=neutral, PublicKeyToken=d353f9ba84f0e281]]
Information: 0 : 2016-10-20T20:32:04.0144450+00:00 0001

INFO: map task memory:7000, update task memory:7000, map task cores:7, update task cores:7,
maxRetry 10, allowedFailedEvaluators 4.

INFO: *** Start time is 10/20/2016 8:32:04 PM
Org.Apache.REEF.Driver.Bridge.Events.EvaluatorRequestor Information: 0 : 2016-10-20T20:32:04.0925881+00:00
INFO: Submitting request for 1 evaluators and 7000 MB memory and  7 core to rack  and runtime
INFO: Allocated Evaluator: container_1475922122639_0016_01_000002, total running running 0
Oct 20, 2016 8:32:06 PM org.apache.reef.javabridge.AllocatedEvaluatorBridge getEvaluatorDescriptorString

INFO: allocated evaluator - serialized evaluator descriptor: IP=xyz, Port=45454, HostName=xyz,
Memory=9216, Core=1, RuntimeName=Yarn
Oct 20, 2016 8:32:08 PM org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl populateNMTokens

This message was sent by Atlassian JIRA

View raw message