airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Douglas Chau <>
Subject Job Submission Limit
Date Mon, 03 Aug 2015 14:50:54 GMT
Hey Devs,

Just wanted to get some input on our plan to implement the queue throttling feature.

Batch Queue Throttling:
- in Orchestrator the current submit() method in GFACPassiveJobSubmitter publishes to rabbitmq
- instead of publishing immediately, we should have the submit method pass the message to
a new component, BatchQueueClass (tentative name), to check when we can unload jobs to submit

Adding BatchQueueClass
- setup a new table(s) to contain computer resource names and their corresponding queues’
current job numbers and maximum job limits
- current data models only have information on max job submission limit for a queue but no
information on how many jobs are currently submitted, so the idea is to implement a increment/decrement
counter in the new table so when a job is submitted, a call to increment the counter will
be made, and when a job is finished a call to decrement the counter will be made
- once that is complete, BatchQueueClass needs to periodically check the new table to see
if the current job number < queue job limit. If it is then we can just pop jobs off to
submit them until we hit the job limit

How does this sound?

View raw message