airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mangirish Wagle <vaglomangir...@gmail.com>
Subject Re: mesos and moving jobs between clusters
Date Tue, 25 Oct 2016 15:36:55 GMT
Hi Mark,

Thanks for your question. So if I understand you correctly, you need kind
of load balancing between identical clusters through a single Mesos master?

With the current setup, from what I understand, we have a separate mesos
masters for every cluster on separate clouds. However, its a good
investigative topic if we can have single mesos master targeting multiple
identical clusters. We have some work ongoing to use a virtual cluster
setup with compute resources across clouds to install mesos, but not sure
if that is what you are looking for.

Regards,
Mangirish




On Tue, Oct 25, 2016 at 11:05 AM, Miller, Mark <mmiller@sdsc.edu> wrote:

> Hi all,
>
>
>
> I posed a question to Suresh (see below), and he asked me to put this
> question on the dev list.
>
> So here it is. I will be grateful for any comments about the issues you
> all are facing, and what has come up in trying this, as
>
> It seems likely that this is a much simpler problem in concept than it is
> in practice, but its solution has many benefits.
>
>
>
> Here is my question:
>
> A group of us have been discussing how we might simplify submitting jobs
> to different compute resources in our current implementation of CIPRES, and
> how cloud computing might facilitate this. But none of us are cloud
> experts. As I understand it, the mesos cluster that I have been seeing in
> the Airavata email threads is intended to make it possible to deploy jobs
> to multiple virtual clusters. I am (we are) wondering if Mesos manages
> submissions to identical virtual clusters on multiple machines, and if that
> works efficiently.
>
>
>
> In our implementation, we have to change the rules to run efficiently on
> different machines, according to gpu availability, and cores per node. I am
> wondering how Mesos/ virtual clusters affect those considerations.
>
> Can mesos create basically identical virtual clusters independent of
> machine?
>
>
> Thanks for any advice.
>
>
>
> Mark
>
>
>
>
>
>
>
>
>

Mime
View raw message