airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miller, Mark" <mmil...@sdsc.edu>
Subject RE: Building a Resource Allocation Service For Airavata
Date Fri, 23 Sep 2016 19:38:40 GMT
Hi Anuj,

Just to follow up on Suresh’s comment.
We do not use an allocation protocol, though Sudhakar does, and probably can help with that.
We do enforce both storage and usage limits on users.

The former is done at login, and users over a certain threshold aren’t allowed to submit
jobs until their storage levels are reduced.
The latter is enforced, but manually in our main site (at least at present).
However, our rest site has a better and more sophisticated method.

Our policy is to allow users to access 50,000 su if they are at a US institution, and 30,000
su if they are at a foreign institution.
When the user hits 10,20, 30, and 50 thousand SU, they receive an email to warn them about
the process.
If they go above, I can turn off their account manually, but I usually ask them how they are
doing first.

In the REST site, each time a user submits a job, we calculate the amount of total SUs it
would consume if it ran to completion.
We subtract this amount from the amount of allocation the user has remaining. If the potential
cost is greater than the amount they have remaining, the submission is rejected. That way,
a user cannot submit 100 jobs each requiring 6800 SU. Which is possible, though it would take
a long time.

The idea of holding a reserve against potential expenses by a running job is not a new idea,
but it prevents things from getting out of hand.

Let me know if you have any questions about how this works.

Best,
Mark



From: Suresh Marru [mailto:smarru@apache.org]
Sent: Friday, September 23, 2016 12:00 PM
To: Airavata Dev <dev@airavata.apache.org>
Subject: Re: Building a Resource Allocation Service For Airavata

Hi Anuj,

This is great summary. Since this is a new capability for Airavata, it will be useful to have
dedicated wiki space to record all information. We can start with a high-level problem description
(you have some of that in the form of use cases), architecture and design thoughts and Implementation
plan (pointers to JIRA’s).

May be you can put it under - https://cwiki.apache.org/confluence/display/AIRAVATA/Architecture+Documentation

Mark and Terri might have some input on this task since CIPRES enforces soft quotas for users
(but I guess there is no formal allocation request process). Once you have the wiki we can
gather the feedback also there.

Thanks,
Suresh

On Sep 23, 2016, at 2:39 PM, Anuj Bhandar <bhandar.anuj@gmail.com<mailto:bhandar.anuj@gmail.com>>
wrote:

Hi Dev,

I have undertaken this project of designing and developing an resource allocation service
for Airavata, to give you some context, this service takes inspiration from XRAS (XSEDE Resource
Allocation Service).

Here is the preliminary design document, which was written as a part of GSOC-2016, https://cwiki.apache.org/confluence/display/AIRAVATA/%5BGsoc+Proposal%5D+Resource+Allocation+Manager+for+Apache+Airavata.

I have derived some use cases by discussing with Sudhakar Pamidighantam ( PI - GridChem),
here is the link : https://issues.apache.org/jira/browse/AIRAVATA-2108.

Any comments and improvements are gladly welcome.

Thanks and best regards,

Anuj Bhandar

Mime
View raw message