ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Onischuk (JIRA)" <>
Subject [jira] [Updated] (AMBARI-18464) Provide Warnings When ulimit Is High To Prevent Heartbeat Lost Issues
Date Mon, 26 Sep 2016 14:30:20 GMT


Andrew Onischuk updated AMBARI-18464:
    Status: Patch Available  (was: Open)

> Provide Warnings When ulimit Is High To Prevent Heartbeat Lost Issues
> ---------------------------------------------------------------------
>                 Key: AMBARI-18464
>                 URL:
>             Project: Ambari
>          Issue Type: Bug
>            Reporter: Andrew Onischuk
>            Assignee: Andrew Onischuk
>             Fix For: 3.0.0
>         Attachments: AMBARI-18464.patch
> Python's `Popen` constructor takes an optional argument called `close_fds`
> which instructs Python to close all open file descriptors except for pipe,
> stdout, and stderr. However, Python's logic chooses to iterate over all
> possible open handles, not just those which are actually open.
>   * With my `ulimit -n 1024`, `Popen` was taking ~2ms
>   * With my `ulimit -n 1000000`, `Popen` was taking ~150ms
> That's an increase of 7400%, and all I did was increase my ulimit. The number
> of FDs opened was consistent.
> In some environments, this `Popen` call can take between 6 and 60 seconds per
> call. This leads to a problem where status commands cannot be drained fast
> enough and results in the agent not responding to heartbeats and not running
> commands.
> This Jira serves two purposes:
>   * Investigate our use of `close_fds` and determine if it's correct (or at least needs
to be parameterized as a configuration option.
>   * Provide a host check warning for the ulimit being too high.

This message was sent by Atlassian JIRA

View raw message