airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pankaj Saha <psa...@binghamton.edu>
Subject Re: [GSoC Proposal] - Integrating Resource Information from Apache Mesos with Apache Airavata’s Job Management Modules
Date Mon, 21 Mar 2016 21:17:45 GMT
Hi Marlon,
Here is the link that I have created.

https://docs.google.com/document/d/1qtFvg4-usT4D_1TDNBsQDFQGZIkH99ideYQ1T3HU9nY/edit?usp=sharing
The draft is created under the GSoC proposal site under Apache foundation
with the same title.


Thanks
Pankaj



On Mon, Mar 21, 2016 at 4:47 PM, Pierce, Marlon <marpierc@iu.edu> wrote:

> Hi Pankaj,
>
> I have some comments, but it would be easier if you created a proposal
> draft in the GSOC site. The google doc option for your draft is better than
> pointing to the Airavata wiki. Please make sure you give comment and
> suggestion permissions.
>
> Marlon
>
>
> From: Pankaj Saha <psaha4@binghamton.edu>
> Reply-To: "dev@airavata.apache.org" <dev@airavata.apache.org>
> Date: Monday, March 21, 2016 at 11:16 AM
> To: dev <dev@airavata.apache.org>
> Subject: [GSoC Proposal] - Integrating Resource Information from Apache
> Mesos with Apache Airavata’s Job Management Modules
>
> Hi Dev Team,
>
> Please review the following GSoC proposal that I plan to submit:
> Title: Integrating Resource Information from Apache Mesos with Apache
> Airavata’s Job Management Modules
>
> Abstract:
> Apache Airavata provides gateway computing capability across clustered
> environments for scientific users. It abstracts away the complexities of
> submitting jobs to HPC platforms and provides users with an intuitive and
> elegant web-based interface to submit jobs. Apache Mesos is a  distributed
> kernel that manages distributed computing resources as a single computer.
> As Airavata is being extended to use Big Data and Cloud tools to launch
> jobs in cloud environments, it needs to retrieve the resource and job
> execution information from the Big Data framework back to the Apache portal
> accessible to the end user. In this project we will develop code and
> scripts to be integrated with the Airavata that will use the HTTP API of
> Mesos to continuously fetch the complete resource and scheduling
> information. This information can then be used by Airavata to dynamically
> monitor and improve its job submission strategy in cloud environments such
> as Jetstream.
>
> Introduction:
> Apache Mesos provides HTTP API endpoints for scheduler, executor, internal
> and admin related queries. To fetch information regarding a clustered
> environment that is managed by the Mesos master, the API can be accessed
> via curl requests over HTTP. The response to such requests will be received
> as well formed json document. We will parse the json response and present
> the information in the format desired. The retrieved information will
> include resource usage, resource available for further jobs, job status,
> time elapsed since the job started, etc.  Airavata, in turn, will use this
> information to determine the resource usage, performance of the jobs on a
> job submission, rapid diagnosis on the health of the submitted jobs.
>
> We will use the observer pattern to continuously pull information from
> Cloud and big Data Resource Managers, such as Apache Mesos, to Airavata.
>
> Any comment and suggestions would be very helpful.
>
> Thanks
> Pankaj
>
>

Mime
View raw message