mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kone" <vinodk...@gmail.com>
Subject Re: Review Request 25035: Fix for MESOS-1688
Date Thu, 11 Sep 2014 05:06:08 GMT


> On Sept. 9, 2014, 7:10 p.m., Vinod Kone wrote:
> > src/master/master.cpp, line 1901
> > <https://reviews.apache.org/r/25035/diff/4/?file=682182#file682182line1901>
> >
> >     I like these warnings.
> >     
> >     Are you planning to get this in to 0.20.1 or 0.21.0 ? If the former, can you
add this to the list of deprecations in CHANGELOG.
> 
> Martin Weindel wrote:
>     Would be nice to see this in 0.20.1.
>     But it is not clear to me, how to update the CHANGELOG. There is no section for upcoming
releases.

Just start one for 0.20.1 and just add the deprecation. See how we did it for 0.20.0 and 0.19.1
for inspiration. As we get close to releasing 0.20.1, the release manager will make sure to
update the CHANGELOG with the tickets and other info.


- Vinod


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25035/#review52763
-----------------------------------------------------------


On Sept. 10, 2014, 10 p.m., Martin Weindel wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25035/
> -----------------------------------------------------------
> 
> (Updated Sept. 10, 2014, 10 p.m.)
> 
> 
> Review request for mesos and Vinod Kone.
> 
> 
> Bugs: MESOS-1688
>     https://issues.apache.org/jira/browse/MESOS-1688
> 
> 
> Repository: mesos-git
> 
> 
> Description
> -------
> 
> As already explained in JIRA MESOS-1688, there are schedulers allocating memory only
for the executor and not for tasks. For tasks only CPU resources are allocated in this case.
> Such a scheduler does not get offered any idle CPUs if the slave has nearly used up all
memory.
> This can easily lead to a dead lock (in the application, not in Mesos).
> 
> Simple example:
> 1. Scheduler allocates all memory of a slave for an executor
> 2. Scheduler launches a task for this executor (allocating 1 CPU)
> 3. Task finishes: 1 CPU , 0 MB memory allocatable.
> 4. No offers are made, as no memory is left. Scheduler will wait for offers forever.
Dead lock in the application.
> 
> To fix this problem, offers must be made if CPU resources are allocatable without considering
allocatable memory
> 
> 
> Diffs
> -----
> 
>   src/common/resources.cpp edf36b1 
>   src/master/constants.cpp faa1503 
>   src/master/hierarchical_allocator_process.hpp 34f8cd6 
>   src/master/master.cpp 18464ba 
>   src/tests/allocator_tests.cpp 774528a 
> 
> Diff: https://reviews.apache.org/r/25035/diff/
> 
> 
> Testing
> -------
> 
> Deployed patched Mesos 0.19.1 on a small cluster with 3 slaves and tested running multiple
parallel Spark jobs in "fine-grained" mode to saturate allocatable memory. The jobs run fine
now. This load always caused a dead lock in all Spark jobs within one minute with the unpatched
Mesos.
> 
> 
> Thanks,
> 
> Martin Weindel
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message