airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anuj Bhandar <bhandar.a...@gmail.com>
Subject Re: Building a Resource Allocation Service For Airavata
Date Fri, 23 Sep 2016 20:20:28 GMT
Hello Mark,

First of all, thank you for the reply !

The design you briefed me, a very good way to control the allocations, 
the insight will definitely help me address few issue's for this service.

I have derived some use-cases after discussing with Sudhakar, Are these 
use-cases exhaustive enough to generalize them to CIPRES, Please review 
and suggest improvements if any.

Use-cases can be found here : 
https://cwiki.apache.org/confluence/display/AIRAVATA/Airavata+Resource+Allocation+Service


Yes Suresh,

I have created a page to document the development phase for this module, 
link is mentioned above.


Thanks and best regards,

Anuj Bhandar


On 09/23/2016 03:38 PM, Miller, Mark wrote:
>
> Hi Anuj,
>
> Just to follow up on Suresh’s comment.
>
> We do not use an allocation protocol, though Sudhakar does, and 
> probably can help with that.
>
> We do enforce both storage and usage limits on users.
>
> The former is done at login, and users over a certain threshold aren’t 
> allowed to submit jobs until their storage levels are reduced.
>
> The latter is enforced, but manually in our main site (at least at 
> present).
>
> However, our rest site has a better and more sophisticated method.
>
> Our policy is to allow users to access 50,000 su if they are at a US 
> institution, and 30,000 su if they are at a foreign institution.
>
> When the user hits 10,20, 30, and 50 thousand SU, they receive an 
> email to warn them about the process.
>
> If they go above, I can turn off their account manually, but I usually 
> ask them how they are doing first.
>
> In the REST site, each time a user submits a job, we calculate the 
> amount of total SUs it would consume if it ran to completion.
>
> We subtract this amount from the amount of allocation the user has 
> remaining. If the potential cost is greater than the amount they have 
> remaining, the submission is rejected. That way, a user cannot submit 
> 100 jobs each requiring 6800 SU. Which is possible, though it would 
> take a long time.
>
> The idea of holding a reserve against potential expenses by a running 
> job is not a new idea, but it prevents things from getting out of hand.
>
> Let me know if you have any questions about how this works.
>
> Best,
>
> Mark
>
> *From:*Suresh Marru [mailto:smarru@apache.org]
> *Sent:* Friday, September 23, 2016 12:00 PM
> *To:* Airavata Dev <dev@airavata.apache.org>
> *Subject:* Re: Building a Resource Allocation Service For Airavata
>
> Hi Anuj,
>
> This is great summary. Since this is a new capability for Airavata, it 
> will be useful to have dedicated wiki space to record all information. 
> We can start with a high-level problem description (you have some of 
> that in the form of use cases), architecture and design thoughts and 
> Implementation plan (pointers to JIRA’s).
>
> May be you can put it under - 
> https://cwiki.apache.org/confluence/display/AIRAVATA/Architecture+Documentation
>
> Mark and Terri might have some input on this task since CIPRES 
> enforces soft quotas for users (but I guess there is no formal 
> allocation request process). Once you have the wiki we can gather the 
> feedback also there.
>
> Thanks,
>
> Suresh
>
>     On Sep 23, 2016, at 2:39 PM, Anuj Bhandar <bhandar.anuj@gmail.com
>     <mailto:bhandar.anuj@gmail.com>> wrote:
>
>     Hi Dev,
>
>     I have undertaken this project of designing and developing an
>     resource allocation service for Airavata, to give you some
>     context, this service takes inspiration from XRAS (XSEDE Resource
>     Allocation Service).
>
>     Here is the preliminary design document, which was written as a
>     part of GSOC-2016,
>     https://cwiki.apache.org/confluence/display/AIRAVATA/%5BGsoc+Proposal%5D+Resource+Allocation+Manager+for+Apache+Airavata.
>
>     I have derived some use cases by discussing with Sudhakar
>     Pamidighantam ( PI - GridChem), here is the link :
>     https://issues.apache.org/jira/browse/AIRAVATA-2108.
>
>     Any comments and improvements are gladly welcome.
>
>     Thanks and best regards,
>
>     Anuj Bhandar
>


Mime
View raw message