hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashwin Sai Shankar <ashan...@netflix.com>
Subject User queues on Fair and Capacity scheduler
Date Mon, 10 Mar 2014 17:02:41 GMT
We have a use case on our clusters where we want
users to be in their own (leaf) queues ie all jobs in a leaf queue belong
to a particular
user so that :
1. We can assign capacity(Cap Sch)/weight(Fair Sch) to individual users.
2. An app in a user-queue can preempt another user from other user-queue if
its capacity
guarantee/fairness is not met. Currently I believe there is no user-based
intra-queue preemption in both the schedulers.

Please let me know if there is any way to configure CS or FS
dynamically(automatically) as new users submit jobs, so that new per user
child queues are created under appropriate parents ?
I'm hoping there is some magic configs that would solve this problem :)


View raw message