helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Craig <mcr...@box.com>
Subject Can Helix balance many small resources on a large cluster?
Date Thu, 10 Nov 2016 23:13:41 GMT
This is related to a use case mentioned in a prior thread:

> .. manage a set of ad-hoc tasks across a cluster of machines. Each task
is a separate resource with 1 partition and 1 replica, intended to run on
exactly 1 node.

Can Helix balance many small resources like this? E.g. 3 resources {R1, R2,
R3} each with 1 partition and 1 replica, on 3 nodes:

N1: {R1}, N2: {R2}, N3: {R3}

Or does Helix balance the resources independently, putting all partitions
on the same node? I.e.

N1: {R1, R2, R3}, N2: {}, N3: {}

I've observed the latter behavior, though I'm wondering if I'm misusing
Helix.

Mime
View raw message