hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Forsberg <forsb...@opera.com>
Subject Re: Scheduling, prioritizing some jobs
Date Tue, 26 Jan 2010 07:25:27 GMT

Simplifying my question: Can I configure Hadoop so that:

*) I have four scheduling pools: highprio-daily, highprio-monthly,
   lowprio-daily, lowprio-monthly.

*) Tasks for jobs that are put in highprio-daily always get priority
   before tasks in highprio-monthly. Highprio-monthly always get
   priority before lowprio-daily, and lowprio-daily always get priority
   before lowprio-monthly?

   If there are several jobs in the same pool, run them in order of
   submission. A job should finish as quickly as possible, so if the
   currently most highly prioritized job needs all task slots, it
   should get them.

Erik Forsberg <forsberg@opera.com>
Developer, Opera Software - http://www.opera.com/

View raw message