mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niklas Nielsen <nik...@mesosphere.io>
Subject Re: multi tenant setup
Date Mon, 04 Aug 2014 16:08:11 GMT
Sorry for the tardy reply. Spark seems to be holding on to those resources
then - are you running Spark in coarse grained mode?
I am not too Spark savvy, but running in fine grained mode should allocate
and free resources on-demand rather than allocating a temporary static
partition up front.
Maybe Tim Chen can chime in here.

Niklas



On 1 August 2014 00:14, Gurvinder Singh <gurvinder.singh@uninett.no> wrote:

> Hi Niklas,
>
> I am using Apache spark with mesos 0.19.1. I have limited resources and
> when I submit a job which takes all of the resources. This is fine as
> when no one is using them, but when one of my colleagues submit his job,
> I would like mesos allows some part of resources assigned to his job
> when part of my jobs are finished, But it seems currently it waits until
> my whole job is finished before starting the other job. Is it due to
> mesos or you think Spark is the one who is blocking the job.
>
> - Gurvinder
> On 07/31/2014 06:12 PM, Niklas Nielsen wrote:
> > Hi Gurvinder,
> >
> > The frameworks competing for resources will get their (weighted) fair
> > share of the cluster. The allocator in the master uses the Dominant
> > Resource Fairness algorithm to do this
> > (http://static.usenix.org/event/nsdi11/tech/full_papers/Ghodsi.pdf).
> > Regarding FIFO, are you referring to 'local' scheduler policies? How
> > tasks are dispatched is up to the individual framework.
> >
> > Cheers,
> > Niklas
> >
> >
> > On 31 July 2014 07:28, Gurvinder Singh <gurvinder.singh@uninett.no
> > <mailto:gurvinder.singh@uninett.no>> wrote:
> >
> >     Hi,
> >
> >     I am wondering how mesos handle the task scheduling when the resource
> >     are limited and multiple users want to access them at the same time.
> Is
> >     there any kind of fair scheduling as I see currently mainly FIFO. If
> >     there is how can I specify that.
> >
> >     Thanks,
> >     Gurvinder
> >
> >
>
>

Mime
View raw message