airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pankaj Saha <psa...@binghamton.edu>
Subject Re: Spinup Mesos-Marathon Cluster for Hybrid Scheduling
Date Mon, 12 Sep 2016 16:26:23 GMT
Hi Shameera,
Are you planning to submits jobs through Marathon framework or Airavata
itself? Is there any new framework module added to submits jobs to Mesos?
Marathon will keep on submitting the same job to cluster even after its
over or error out. Apache Aurora could be better option what do you think?
Also Aurora comes with its own executor which could be an advantage over
Marathon.

Thanks
Pankaj

On Mon, Sep 12, 2016 at 11:19 AM, Shameera Rathnayaka <
shameerainfo@gmail.com> wrote:

> Hi Dev,
>
> The effort of getting use Cloud infrastructure to run MPI and BigData jobs
> using Airavata, we use Apache Mesos as  resource allocation framework to
> manage different type of clusters (i.e HPC node cluster to run MPI jobs and
> spark, Hadoop big data clusters to run bigdata applications). I came up
> with Ansible script to spin up Mesos cluster on the target set of nodes.
> You can find the script here https://github.com/shamrath/mesos-deployment
> I am thinking of  move this code to Airavata if all agreed. I would happy
> to answer any question related to this.
>
> Thanks,
> Shameera.
> --
> Shameera Rathnayaka
>

Mime
View raw message