ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Onischuk (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-18464) Provide Warnings When ulimit Is High To Prevent Heartbeat Lost Issues
Date Mon, 26 Sep 2016 14:30:20 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-18464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Onischuk updated AMBARI-18464:
-------------------------------------
    Status: Patch Available  (was: Open)

> Provide Warnings When ulimit Is High To Prevent Heartbeat Lost Issues
> ---------------------------------------------------------------------
>
>                 Key: AMBARI-18464
>                 URL: https://issues.apache.org/jira/browse/AMBARI-18464
>             Project: Ambari
>          Issue Type: Bug
>            Reporter: Andrew Onischuk
>            Assignee: Andrew Onischuk
>             Fix For: 3.0.0
>
>         Attachments: AMBARI-18464.patch
>
>
> Python's `Popen` constructor takes an optional argument called `close_fds`
> which instructs Python to close all open file descriptors except for pipe,
> stdout, and stderr. However, Python's logic chooses to iterate over all
> possible open handles, not just those which are actually open.
>   * With my `ulimit -n 1024`, `Popen` was taking ~2ms
>   * With my `ulimit -n 1000000`, `Popen` was taking ~150ms
> That's an increase of 7400%, and all I did was increase my ulimit. The number
> of FDs opened was consistent.
> In some environments, this `Popen` call can take between 6 and 60 seconds per
> call. This leads to a problem where status commands cannot be drained fast
> enough and results in the agent not responding to heartbeats and not running
> commands.
> This Jira serves two purposes:
>   * Investigate our use of `close_fds` and determine if it's correct (or at least needs
to be parameterized as a configuration option.
>   * Provide a host check warning for the ulimit being too high.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message