mesos-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anand Mazumdar" <mazumdar.an...@gmail.com>
Subject Re: Review Request 42053: Add flags to set size of completed task/framework history.
Date Fri, 08 Jan 2016 07:16:06 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42053/#review113428
-----------------------------------------------------------


LGTM. Two minor issues:

1. May be we can escape without passing the `Master` pointer around?
2. Would be great to have a test to go with this?


src/master/master.hpp (line 1356)
<https://reviews.apache.org/r/42053/#comment174051>

    hmmm.. Do we need to pass an pointer to `Master` here? Looks like a overkill to me.
    
    Can't we either pass the `Flags` object or just pass on the value of `flags.max_completed_frameworks`
here?



src/master/master.hpp (line 1696)
<https://reviews.apache.org/r/42053/#comment174052>

    Ditto as above. What do you think about just passing the `size_t` directly here?
    
    If not, let's do: (since we already have the `Master` here as an argument)
    s/`_master->`/`master->`, the `master` member would already have been initialized
by now.


- Anand Mazumdar


On Jan. 8, 2016, 1:29 a.m., Kevin Klues wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42053/
> -----------------------------------------------------------
> 
> (Updated Jan. 8, 2016, 1:29 a.m.)
> 
> 
> Review request for mesos and Ben Mahler.
> 
> 
> Bugs: MESOS-3307
>     https://issues.apache.org/jira/browse/MESOS-3307
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> The default size of the buffers used to hold the state of completed
> tasks/frameworks is very large. However, many frameworks don't care much
> about this information when requesting a master's state. Moreover, if a
> large number of frameworks request this state simultaneously, the
> master can quickly become overwhelmed because the process of generating
> this state both blocks the master and takes up a lot of cycles. By
> allowing the master to configure the size of the buffers used to hold
> this state, we give it the power to significantly reduce the amount of
> state it needs to maintain.
> 
> This change allows the master to limit the size of this state via
> command line flags.
> 
> This commit is based on a pull request generated by Felix Bechstein at:
> https://github.com/apache/mesos/pull/82
> 
> 
> Diffs
> -----
> 
>   src/master/constants.hpp f1624e1ed5091f08f57d4382b344cab8b05ba840 
>   src/master/constants.cpp 589f2ee55a24d7e8b9352a4c8f94a3785fd1bef4 
>   src/master/flags.hpp 9af6c68eef6bcf39d5776809fab6c66dc95da6b2 
>   src/master/flags.cpp f864419a6276356c97a0a4fe29e5cfce6c4660c4 
>   src/master/master.hpp f764915ff8bbf74b99a617e3c4735d38fc13e5f0 
>   src/master/master.cpp 145c5932610bd019eb090947569b608df6815c3a 
> 
> Diff: https://reviews.apache.org/r/42053/diff/
> 
> 
> Testing
> -------
> 
> On Darwin I launched a master with:
> 
> ./bin/mesos-master.sh --ip=127.0.0.1 --work_dir=/var/lib/mesos --max_completed_tasks_per_framework=2
--max_completed_frameworks=1
> 
> and a slave with:
> 
> ./bin/mesos-slave.sh --master=127.0.0.1:5050
> 
> and then ran a bunch of instances of:
> 
> ./src/test-framework --master=127.0.0.1:5050
> 
> each of which runs 5 tasks to completion
> 
> I then ran:
> 
> curl http://localhost:5050/tasks
> 
> and verified that only 1 framework and 2 of its completed tasks were given back to me
in the json that was returned.  I repeated this for a number of other configurations with
max_completed_frameworks=0..2 and max_completed_tasks_per_framework=0..5 and verified visually
that the proper number of tasks/frameworks were being returned by the /tasks endpoint.
> 
> 
> Thanks,
> 
> Kevin Klues
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message