airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Douglas Chau <>
Subject GSoC project
Date Sun, 14 Jun 2015 17:44:08 GMT

For my GSoC project I want to tackle the issue of cluster recommendation for airavata users;
I will be working with John, another GSoC student, on this component. The current PGA interface
forces a user to pick a specific resource, but of course, as an end user, they may likely
have no idea as to which resource to choose. The end goal is to provide a tip/hint in the
user interface that points to the cluster that will be most “optimal” for the job; however,
users will still be allowed to select a resource of their choice regardless of recommendation.

The cluster recommendation component will consist of two metrics (these categories can be
further fleshed out): availability and runtime benchmarks. Availability will be determined
by the cluster health, queue prediction times, total allocations (SU’s), filesystem availability,
etc. Most of our clusters now are under XSEDE, so this information can be queried from their
API. For other clusters outside XSEDE we will have to use other methods to send back information
(i.e. manually ssh-ing into a cluster to grab information). But, for now I am going to focus
on the grabbing information from the XSEDE clusters. Runtime benchmarking falls under John’s
GSoC project, but the idea is beyond availability of a cluster, we also need to consider which
cluster is historically known to run specific jobs faster; this is a more challenging problem.
In the end, these two metrics will be evaluated to give a recommended cluster for a specific

All this cluster information will be stored in registry. Then we can add a new airavata API
method to query and crunch this cluster information from the registry. After a recommendation
is determined, the information will return to the PGA client and populate the page with the
recommendation information.

This is a quick overview of what I want to do and there are probably more details that I may
have left out, so if there are any questions please ask! Also, tips or advice on how to tackle
this problem would be appreciated.

View raw message