helix-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinoth Chandar <vin...@uber.com>
Subject Balancing out skews in FULL_AUTO mode with built-in rebalancer
Date Tue, 15 Mar 2016 20:45:21 GMT
Hi guys,

We are hitting a fairly known issue where we have 100s of resource with < 8
resources spreading across 10 servers and the built-in assignment always
assigns partitions from first to last, resulting in heavy skew for a few
nodes.

Chatted with Kishore offline and made a patch as here
<https://gist.github.com/vinothchandar/e8837df301501f85e257>.Tested with 5
resources with 2 partitions each across 8 servers, logging out the
nodeShift & ultimate index picked does indicate that we choose servers
other than the first two, which is good

But
1) I am guessing it gets overriden by other logic in
computePartitionAssignment(..), the end assignment is still skewed.
2) Even with murmur hash, there is some skew on the nodeshift, which needs
to ironed out.

I will keep chipping at this.. Any feedback appreciated

Thanks
Vinoth

Mime
View raw message