hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mehmet Belgin <mehmet.bel...@oit.gatech.edu>
Subject Is there a way to limit # of hadoop tasks per user at runtime?
Date Tue, 21 May 2013 21:43:27 GMT
Hi Everyone,

I was wondering if there is a way for limiting the number of tasks (map+reduce) *per user*
at runtime? Using an environment variable perhaps? I am asking this from a resource provisioning
perspective. I am trying to come up with a N-token licensing system for multiple users to
use our limited hadoop resources simultaneously. That is, when user A checks out 6 tokens,
 he/she can only run 6 hadoop tasks. 

If there is no such thing in hadoop, has anyone tried to integrate hadoop with torque/moab
(or any other RM or scheduler)? Any advice in that direction will be appreciated :)

Thanks in advance,

View raw message