ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Olivér Szabó (JIRA) <j...@apache.org>
Subject [jira] [Updated] (AMBARI-14690) Configurable system resource values for ambari-agent
Date Sat, 23 Jan 2016 19:15:39 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-14690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Olivér Szabó updated AMBARI-14690:
----------------------------------
    Description: 
When an ambari agent starts, host system details are registered into ambari server database.
These values are calculated by ambari-agents based on different files on the hosts (e.g :
/proc/meminfo). 

In some cases it isn't a correct behavior: If ambari-agent is in a docker-container, it will
see the same memory/cpu details. (stack advisor also uses these values, its possible stack
advisor can recommend a too high memory value for some services)

Solution: Configurable system resources for hosts (ambari-agent).
Added new option to 'agent' section in ambari-agent.ini: system_resource_overrides. It points
to a directory, if the value/directory does not exist or empty, it will use the default behavior

{code:java}
[agent]
...
system_resource_overrides=/etc/resource_overrides
...
{code}
It reads all json files from that directory. JSON structure looks like this:
{code:java}
{
"memoryfree" :  "15600",
"memorytotal" : ....
}
{code}

In ambari-agent Facter.py set these values, all of the factor info values can be redefined
in one or multiple JSON files.
That means ambari is not responsible to gather these values from the system.

- use case: 
During 'docker run' , before 'ambari-agent start', 'system_resource_overrides' value can be
overwritten, or the default ('/etc/resource_overrides') can be mount with '-v' flag (docker
run -v /my/path:/etc/resource_overrides).

Facter.py is called during ambari-agent start, so if the values will be modified later, ambari-agent
needs to be restarted.



  was:
When an ambari agent starts, host system details are registered into ambari server database.
These values are calculated by ambari-agents based on different files on the hosts (e.g :
/proc/meminfo). 

In some cases it isn't a correct behavior: If ambari-agent is in a docker-container, it will
see the same memory/cpu details. (stack advisor also uses these values, its possible stack
advisor can recommend a too high memory value for some services)

Solution: Configurable system resources in ambari-agent.ini
Added new option to 'agent' section: system_resource_overrides. It points to a directory,
if the value/directory does not exist or empty, it will use the default behavior

{code:java}
[agent]
...
system_resource_overrides=/etc/resource_overrides
...
{code}

{code:java}

{code}

In ambari-agent Facter.py set these values, all of the factor info values can be redefined
in ambari-agent.ini file.

That means ambari is not responsible to gather these values from the system.

- use case: 
During 'docker run' , if there is a script which calls 'ambari-agent start', before that,
some of the memory values can be replaced/inserted into the ambari-agent.ini file in that
script. (e.g. the user knows that, 4 ambari-agent containers will be started, with the same
services, so the real memory needs to be devided by 4 etc.)

Facter.py is called during ambari-agent start, so if the values will be modified later, ambari-agent
needs to be restarted.




> Configurable system resource values for ambari-agent
> ----------------------------------------------------
>
>                 Key: AMBARI-14690
>                 URL: https://issues.apache.org/jira/browse/AMBARI-14690
>             Project: Ambari
>          Issue Type: Improvement
>          Components: ambari-agent
>    Affects Versions: 2.2.0
>            Reporter: Olivér Szabó
>            Assignee: Olivér Szabó
>             Fix For: 2.2.2
>
>         Attachments: AMBARI-14690.patch, AMBARI-14690_v4.patch
>
>
> When an ambari agent starts, host system details are registered into ambari server database.
These values are calculated by ambari-agents based on different files on the hosts (e.g :
/proc/meminfo). 
> In some cases it isn't a correct behavior: If ambari-agent is in a docker-container,
it will see the same memory/cpu details. (stack advisor also uses these values, its possible
stack advisor can recommend a too high memory value for some services)
> Solution: Configurable system resources for hosts (ambari-agent).
> Added new option to 'agent' section in ambari-agent.ini: system_resource_overrides. It
points to a directory, if the value/directory does not exist or empty, it will use the default
behavior
> {code:java}
> [agent]
> ...
> system_resource_overrides=/etc/resource_overrides
> ...
> {code}
> It reads all json files from that directory. JSON structure looks like this:
> {code:java}
> {
> "memoryfree" :  "15600",
> "memorytotal" : ....
> }
> {code}
> In ambari-agent Facter.py set these values, all of the factor info values can be redefined
in one or multiple JSON files.
> That means ambari is not responsible to gather these values from the system.
> - use case: 
> During 'docker run' , before 'ambari-agent start', 'system_resource_overrides' value
can be overwritten, or the default ('/etc/resource_overrides') can be mount with '-v' flag
(docker run -v /my/path:/etc/resource_overrides).
> Facter.py is called during ambari-agent start, so if the values will be modified later,
ambari-agent needs to be restarted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message