mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From haosdent <haosd...@gmail.com>
Subject Re: Setting ulimits on mesos-slave
Date Tue, 26 Apr 2016 16:43:14 GMT
I disable the limit globally and retry set

```
limit fsize unlimited unlimited
```

in /etc/init/mesos-slave.conf after reboot. It works as well according to
check limits under /proc May you double check this in your side? I wonder
if the fszie only effective on Mesos Agent and doesn't work in your Mesos
executor.

On Tue, Apr 26, 2016 at 8:55 PM, June Taylor <june@umn.edu> wrote:

> Perhaps the "unlimited" keyword is what is not working? What should be
> specified for Unlimited?
>
>
> Thanks,
> June Taylor
> System Administrator, Minnesota Population Center
> University of Minnesota
>
> On Mon, Apr 25, 2016 at 9:48 PM, haosdent <haosdent@gmail.com> wrote:
>
>> According my test, it works in my side.
>>
>> * Before add it to /etc/init/mesos-slave.conf
>>
>> ```
>> cat /proc/16550/limits
>> Limit                     Soft Limit           Hard Limit           Units
>> Max cpu time              unlimited            unlimited
>>  seconds
>> Max file size             unlimited            unlimited            bytes
>> Max data size             unlimited            unlimited            bytes
>> Max stack size            10485760             unlimited            bytes
>> Max core file size        0                    unlimited            bytes
>> Max resident set          unlimited            unlimited            bytes
>> Max processes             63655                63655
>>  processes
>> Max open files            8192                 8192                 files
>> Max locked memory         65536                65536                bytes
>> Max address space         unlimited            unlimited            bytes
>> Max file locks            unlimited            unlimited            locks
>> Max pending signals       63655                63655
>>  signals
>> Max msgqueue size         819200               819200               bytes
>> Max nice priority         0                    0
>> Max realtime priority     0                    0
>> Max realtime timeout      unlimited            unlimited            us
>> ```
>>
>> * After add it to /etc/init/mesos-slave.conf
>>
>> ```
>> description "mesos slave"
>>
>> # I didn't use ulimit because I have already set ulimit globally.
>> limit fsize 20001 20001
>> limit nofile 20000 20000
>> ```
>>
>> ```
>> cat /proc/16602/limits
>> Limit                     Soft Limit           Hard Limit           Units
>> Max cpu time              unlimited            unlimited
>>  seconds
>> Max file size             20001                20001                bytes
>>   # <- Have changed
>> Max data size             unlimited            unlimited            bytes
>> Max stack size            10485760             unlimited            bytes
>> Max core file size        0                    unlimited            bytes
>> Max resident set          unlimited            unlimited            bytes
>> Max processes             63655                63655
>>  processes
>> Max open files            8192                 8192                 files
>> Max locked memory         65536                65536                bytes
>> Max address space         unlimited            unlimited            bytes
>> Max file locks            unlimited            unlimited            locks
>> Max pending signals       63655                63655
>>  signals
>> Max msgqueue size         819200               819200               bytes
>> Max nice priority         0                    0
>> Max realtime priority     0                    0
>> Max realtime timeout      unlimited            unlimited            us
>> ```
>>
>> On Tue, Apr 26, 2016 at 3:51 AM, June Taylor <june@umn.edu> wrote:
>>
>>> Hello. We are running it as root, and it is able to specify the ulimit
>>> for open files, as noted in our config file, but it is not setting the File
>>> Size limit, which is remaining at 8MB.
>>>
>>>
>>> Thanks,
>>> June Taylor
>>> System Administrator, Minnesota Population Center
>>> University of Minnesota
>>>
>>> On Mon, Apr 25, 2016 at 2:50 PM, Dick Davies <dick@hellooperator.net>
>>> wrote:
>>>
>>>> Hi June
>>>>
>>>> are you running Mesos as root, or a non-privileged user? Non-root
>>>> won't be able to up their own ulimit too high
>>>> (sorry, not an upstart expert as RHELs is laughably incomplete).
>>>>
>>>> On 25 April 2016 at 19:15, June Taylor <june@umn.edu> wrote:
>>>> > What I'm saying is even putting them within the upstart script, per
>>>> the
>>>> > Mesos documentation, isn't working for the file block limit. We're
>>>> still
>>>> > getting 8MB useable, and as a result executors fail when attempting
>>>> to write
>>>> > larger files.
>>>> >
>>>> >
>>>> > Thanks,
>>>> > June Taylor
>>>> > System Administrator, Minnesota Population Center
>>>> > University of Minnesota
>>>> >
>>>> > On Mon, Apr 25, 2016 at 11:53 AM, haosdent <haosdent@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> If you set in your upstart script, it isn't system wide and only
>>>> effective
>>>> >> in that session. I think need change /etc/security/limits.conf and
>>>> >> /etc/sysctl.conf to make your ulimit work globally.
>>>> >>
>>>> >> On Tue, Apr 26, 2016 at 12:43 AM, June Taylor <june@umn.edu>
wrote:
>>>> >>>
>>>> >>> Somewhere an 8MB maximum file size is being applied on just
one of
>>>> our
>>>> >>> slaves, for example.
>>>> >>>
>>>> >>>
>>>> >>> Thanks,
>>>> >>> June Taylor
>>>> >>> System Administrator, Minnesota Population Center
>>>> >>> University of Minnesota
>>>> >>>
>>>> >>> On Mon, Apr 25, 2016 at 11:42 AM, June Taylor <june@umn.edu>
wrote:
>>>> >>>>
>>>> >>>> We are operating a 6-node cluster running on Ubuntu, and
have
>>>> noticed
>>>> >>>> that the ulimit settings within the slave context are difficult
to
>>>> set and
>>>> >>>> predict.
>>>> >>>>
>>>> >>>> The documentation is a bit unclear on this point, as well.
>>>> >>>>
>>>> >>>> We have had some luck adding a configuration line to
>>>> >>>> /etc/init/mesos-slave.conf as follows:
>>>> >>>> limit nofile 20000 20000
>>>> >>>> limit fsize unlimited unlimited
>>>> >>>>
>>>> >>>> The nofile limit seems to be respected, however the fsize
limit
>>>> does
>>>> >>>> not.
>>>> >>>>
>>>> >>>> It is also mysterious that the system-wide limits are not
>>>> inherited by
>>>> >>>> the slave process. We would prefer to set all of these system-wide
>>>> and have
>>>> >>>> mesos-slave observe them.
>>>> >>>>
>>>> >>>> Can you please advise where you are setting your ulimits
for the
>>>> >>>> mesos-slave if it is working for you?
>>>> >>>>
>>>> >>>> Thanks,
>>>> >>>> June Taylor
>>>> >>>> System Administrator, Minnesota Population Center
>>>> >>>> University of Minnesota
>>>> >>>
>>>> >>>
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Best Regards,
>>>> >> Haosdent Huang
>>>> >
>>>> >
>>>>
>>>
>>>
>>
>>
>> --
>> Best Regards,
>> Haosdent Huang
>>
>
>


-- 
Best Regards,
Haosdent Huang

Mime
View raw message