Mailing-List: contact user-help@giraph.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@giraph.apache.org
MIME-Version: 1.0
From: Ravikant Dindokar <ravikant.iisc@gmail.com>
Date: Mon, 27 Nov 2017 11:24:10 +0530
Message-ID: <CA+fHtazD1ESrJE8BiWrpfsqTTxZcXnuyT+F5-my=jQ8m11npzQ@mail.gmail.com>
Subject: Worker to task id mapping in giraph
To: giraph-users@googlegroups.com, user@giraph.apache.org
Content-Type: multipart/alternative; boundary="94eb2c0e420a0d85c8055ef088bd"
archived-at: Mon, 27 Nov 2017 05:54:16 -0000

--94eb2c0e420a0d85c8055ef088bd
Content-Type: text/plain; charset="UTF-8"

Hi,

I am trying to change the mapping of partitions to workers in

createInitialPartitionOwners(Collection<WorkerInfo> availableWorkerInfos,
int maxWorkers) defined
in org.apache.giraph.partition.MasterGraphPartitionerImpl.java.

The first argument to this method is the list of available workers and each
worker has a task id associated with it.

When I am specifying the number of workers as 40, the task id are 1 to 40.

I have removed one node from the yarn-cluster by following this answer
https://stackoverflow.com/questions/41820759/how-to-separately-specify-a-set-of-nodes-for-hdfs-and-others-for-mapreduce-jobs
.

After this change, the task ids assigned to the workers are no longer
consecutive (e.g. for 40 workers the range of task id is 1- 43). Some
task-ids are excluded.

Is there any way to get these task ids strictly consecutive?

Thanks
Ravikant

--94eb2c0e420a0d85c8055ef088bd
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi,<div><br></div><div>I am trying to change the mapping o=
f partitions to workers in=C2=A0</div><div><br></div><div>createInitialPart=
itionOwners(Collection&lt;WorkerInfo&gt; availableWorkerInfos, int maxWorke=
rs) defined in=C2=A0org.apache.giraph.partition.MasterGraphPartitionerImpl.=
java.<br></div><div><br></div><div>The first argument to this method is the=
 list of available workers and each worker has a task id associated with it=
.</div><div><br></div><div>When I am specifying the number of workers as 40=
, the task id are 1 to 40.</div><div><br></div><div>I have removed one node=
 from the yarn-cluster by following this answer=C2=A0<a href=3D"https://sta=
ckoverflow.com/questions/41820759/how-to-separately-specify-a-set-of-nodes-=
for-hdfs-and-others-for-mapreduce-jobs">https://stackoverflow.com/questions=
/41820759/how-to-separately-specify-a-set-of-nodes-for-hdfs-and-others-for-=
mapreduce-jobs</a>.</div><div><br></div><div>After this change, the task id=
s assigned to the workers are no longer consecutive (e.g. for 40 workers th=
e range of task id is 1- 43). Some task-ids are excluded.=C2=A0</div><div><=
br></div><div>Is there any way to get these task ids strictly consecutive?=
=C2=A0</div><div><br></div><div>Thanks</div><div>Ravikant=C2=A0</div></div>

--94eb2c0e420a0d85c8055ef088bd--