hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcin Tustin <mtus...@handybook.com>
Subject Configuring capacity scheduling in the face of YARN-3216
Date Wed, 06 Apr 2016 18:31:06 GMT
Many people will be aware of YARN-3216 (
https://issues.apache.org/jira/browse/YARN-3216). In short, the capacity
scheduler calculates all values only off the default label and queue.

My colleague Dave and I (and really mostly Dave!) configured our cluster to
work pretty much as desired even with that bug extant. I've written up how
we did it here:
https://medium.com/handy-tech/practical-capacity-scheduling-with-yarn-28548ae4fb88#.5ihha0oqy

The short version is: keep the bulk of capacity under the default
queue/node and get a bit clever with calculating limits.

Marcin

-- 
Want to work at Handy? Check out our culture deck and open roles 
<http://www.handy.com/careers>
Latest news <http://www.handy.com/press> at Handy
Handy just raised $50m 
<http://venturebeat.com/2015/11/02/on-demand-home-service-handy-raises-50m-in-round-led-by-fidelity/>
led 
by Fidelity


Mime
View raw message