ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Onischuk (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-16162) Reduce NN start time by removing redundant haadmin calls.
Date Thu, 28 Apr 2016 15:23:12 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-16162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Onischuk updated AMBARI-16162:
-------------------------------------
    Description: 
I found this while playing around with one of the clusters

    
    
    2016-04-27 14:21:50,062 - HdfsResource['/tmp'] {'security_enabled': False, 'only_if':
'ambari-sudo.sh su hdfs -l -s /bin/bash -c \'export  PATH=\'"\'"\'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin\'"\'"\'
; /usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs --config /usr/hdp/current/hadoop-client/conf
haadmin -getServiceState nn2 | grep active\'', 'keytab': [EMPTY], 'hadoop_bin_dir': '/usr/hdp/current/hadoop-client/bin',
'default_fs': 'wasb://hwx-test-hdp24-hadoop@storagewuteam02.blob.core.windows.net', 'hdfs_resource_ignore_file':
'/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local':
'/usr/bin/kinit', 'principal_name': None, 'user': 'hdfs', 'owner': 'hdfs', 'hadoop_conf_dir':
'/usr/hdp/current/hadoop-client/conf', 'type': 'directory', 'action': ['create_on_execute'],
'mode': 0777}
    2016-04-27 14:21:58,266 - Skipping 'HdfsResource['/tmp']' because it is in ignore file
/var/lib/ambari-agent/data/.hdfs_resource_ignore.
    2016-04-27 14:21:58,267 - HdfsResource['/user/ambari-qa'] {'security_enabled': False,
'only_if': 'ambari-sudo.sh su hdfs -l -s /bin/bash -c \'export  PATH=\'"\'"\'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin\'"\'"\'
; /usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs --config /usr/hdp/current/hadoop-client/conf
haadmin -getServiceState nn2 | grep active\'', 'keytab': [EMPTY], 'hadoop_bin_dir': '/usr/hdp/current/hadoop-client/bin',
'default_fs': 'wasb://hwx-test-hdp24-hadoop@storagewuteam02.blob.core.windows.net', 'hdfs_resource_ignore_file':
'/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local':
'/usr/bin/kinit', 'principal_name': None, 'user': 'hdfs', 'owner': 'ambari-qa', 'hadoop_conf_dir':
'/usr/hdp/current/hadoop-client/conf', 'type': 'directory', 'action': ['create_on_execute'],
'mode': 0770}
    2016-04-27 14:22:05,522 - Skipping 'HdfsResource['/user/ambari-qa']' because it is in
ignore file /var/lib/ambari-agent/data/.hdfs_resource_ignore.
    2016-04-27 14:22:05,523 - HdfsResource[None] {'security_enabled': False, 'only_if': 'ambari-sudo.sh
su hdfs -l -s /bin/bash -c \'export  PATH=\'"\'"\'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin\'"\'"\'
; /usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs --config /usr/hdp/current/hadoop-client/conf
haadmin -getServiceState nn2 | grep active\'', 'keytab': [EMPTY], 'hadoop_bin_dir': '/usr/hdp/current/hadoop-client/bin',
'default_fs': 'wasb://hwx-test-hdp24-hadoop@storagewuteam02.blob.core.windows.net', 'hdfs_resource_ignore_file':
'/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local':
'/usr/bin/kinit', 'principal_name': None, 'user': 'hdfs', 'action': ['execute'], 'hadoop_conf_dir':
'/usr/hdp/current/hadoop-client/conf'}
    2016-04-27 14:22:13,846 - No resources to create. 'create_on_execute' or 'delete_on_execute'
wasn't triggered before this 'execute' action.
    2016-04-27 14:22:13,846 - Ranger admin not installed
    

Time 14:21:50 - 14:22:13 was spend just re-running haadmin calls, while we
already figured

This should give ~15s time reduce.



  was:
I found this while playing around with Humbold cluster:

    
    
    2016-04-27 14:21:50,062 - HdfsResource['/tmp'] {'security_enabled': False, 'only_if':
'ambari-sudo.sh su hdfs -l -s /bin/bash -c \'export  PATH=\'"\'"\'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin\'"\'"\'
; /usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs --config /usr/hdp/current/hadoop-client/conf
haadmin -getServiceState nn2 | grep active\'', 'keytab': [EMPTY], 'hadoop_bin_dir': '/usr/hdp/current/hadoop-client/bin',
'default_fs': 'wasb://hwx-test-hdp24-hadoop@storagewuteam02.blob.core.windows.net', 'hdfs_resource_ignore_file':
'/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local':
'/usr/bin/kinit', 'principal_name': None, 'user': 'hdfs', 'owner': 'hdfs', 'hadoop_conf_dir':
'/usr/hdp/current/hadoop-client/conf', 'type': 'directory', 'action': ['create_on_execute'],
'mode': 0777}
    2016-04-27 14:21:58,266 - Skipping 'HdfsResource['/tmp']' because it is in ignore file
/var/lib/ambari-agent/data/.hdfs_resource_ignore.
    2016-04-27 14:21:58,267 - HdfsResource['/user/ambari-qa'] {'security_enabled': False,
'only_if': 'ambari-sudo.sh su hdfs -l -s /bin/bash -c \'export  PATH=\'"\'"\'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin\'"\'"\'
; /usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs --config /usr/hdp/current/hadoop-client/conf
haadmin -getServiceState nn2 | grep active\'', 'keytab': [EMPTY], 'hadoop_bin_dir': '/usr/hdp/current/hadoop-client/bin',
'default_fs': 'wasb://hwx-test-hdp24-hadoop@storagewuteam02.blob.core.windows.net', 'hdfs_resource_ignore_file':
'/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local':
'/usr/bin/kinit', 'principal_name': None, 'user': 'hdfs', 'owner': 'ambari-qa', 'hadoop_conf_dir':
'/usr/hdp/current/hadoop-client/conf', 'type': 'directory', 'action': ['create_on_execute'],
'mode': 0770}
    2016-04-27 14:22:05,522 - Skipping 'HdfsResource['/user/ambari-qa']' because it is in
ignore file /var/lib/ambari-agent/data/.hdfs_resource_ignore.
    2016-04-27 14:22:05,523 - HdfsResource[None] {'security_enabled': False, 'only_if': 'ambari-sudo.sh
su hdfs -l -s /bin/bash -c \'export  PATH=\'"\'"\'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin\'"\'"\'
; /usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs --config /usr/hdp/current/hadoop-client/conf
haadmin -getServiceState nn2 | grep active\'', 'keytab': [EMPTY], 'hadoop_bin_dir': '/usr/hdp/current/hadoop-client/bin',
'default_fs': 'wasb://hwx-test-hdp24-hadoop@storagewuteam02.blob.core.windows.net', 'hdfs_resource_ignore_file':
'/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local':
'/usr/bin/kinit', 'principal_name': None, 'user': 'hdfs', 'action': ['execute'], 'hadoop_conf_dir':
'/usr/hdp/current/hadoop-client/conf'}
    2016-04-27 14:22:13,846 - No resources to create. 'create_on_execute' or 'delete_on_execute'
wasn't triggered before this 'execute' action.
    2016-04-27 14:22:13,846 - Ranger admin not installed
    

Time 14:21:50 - 14:22:13 was spend just re-running haadmin calls, while we
already figured

This should give ~15s time reduce for Humboldt.




> Reduce NN start time by removing redundant haadmin calls. 
> ----------------------------------------------------------
>
>                 Key: AMBARI-16162
>                 URL: https://issues.apache.org/jira/browse/AMBARI-16162
>             Project: Ambari
>          Issue Type: Bug
>            Reporter: Andrew Onischuk
>            Assignee: Andrew Onischuk
>             Fix For: 2.4.0
>
>         Attachments: AMBARI-16162.patch
>
>
> I found this while playing around with one of the clusters
>     
>     
>     2016-04-27 14:21:50,062 - HdfsResource['/tmp'] {'security_enabled': False, 'only_if':
'ambari-sudo.sh su hdfs -l -s /bin/bash -c \'export  PATH=\'"\'"\'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin\'"\'"\'
; /usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs --config /usr/hdp/current/hadoop-client/conf
haadmin -getServiceState nn2 | grep active\'', 'keytab': [EMPTY], 'hadoop_bin_dir': '/usr/hdp/current/hadoop-client/bin',
'default_fs': 'wasb://hwx-test-hdp24-hadoop@storagewuteam02.blob.core.windows.net', 'hdfs_resource_ignore_file':
'/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local':
'/usr/bin/kinit', 'principal_name': None, 'user': 'hdfs', 'owner': 'hdfs', 'hadoop_conf_dir':
'/usr/hdp/current/hadoop-client/conf', 'type': 'directory', 'action': ['create_on_execute'],
'mode': 0777}
>     2016-04-27 14:21:58,266 - Skipping 'HdfsResource['/tmp']' because it is in ignore
file /var/lib/ambari-agent/data/.hdfs_resource_ignore.
>     2016-04-27 14:21:58,267 - HdfsResource['/user/ambari-qa'] {'security_enabled': False,
'only_if': 'ambari-sudo.sh su hdfs -l -s /bin/bash -c \'export  PATH=\'"\'"\'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin\'"\'"\'
; /usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs --config /usr/hdp/current/hadoop-client/conf
haadmin -getServiceState nn2 | grep active\'', 'keytab': [EMPTY], 'hadoop_bin_dir': '/usr/hdp/current/hadoop-client/bin',
'default_fs': 'wasb://hwx-test-hdp24-hadoop@storagewuteam02.blob.core.windows.net', 'hdfs_resource_ignore_file':
'/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local':
'/usr/bin/kinit', 'principal_name': None, 'user': 'hdfs', 'owner': 'ambari-qa', 'hadoop_conf_dir':
'/usr/hdp/current/hadoop-client/conf', 'type': 'directory', 'action': ['create_on_execute'],
'mode': 0770}
>     2016-04-27 14:22:05,522 - Skipping 'HdfsResource['/user/ambari-qa']' because it is
in ignore file /var/lib/ambari-agent/data/.hdfs_resource_ignore.
>     2016-04-27 14:22:05,523 - HdfsResource[None] {'security_enabled': False, 'only_if':
'ambari-sudo.sh su hdfs -l -s /bin/bash -c \'export  PATH=\'"\'"\'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/usr/hdp/current/hadoop-client/bin\'"\'"\'
; /usr/hdp/current/hadoop-hdfs-namenode/bin/hdfs --config /usr/hdp/current/hadoop-client/conf
haadmin -getServiceState nn2 | grep active\'', 'keytab': [EMPTY], 'hadoop_bin_dir': '/usr/hdp/current/hadoop-client/bin',
'default_fs': 'wasb://hwx-test-hdp24-hadoop@storagewuteam02.blob.core.windows.net', 'hdfs_resource_ignore_file':
'/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site': ..., 'kinit_path_local':
'/usr/bin/kinit', 'principal_name': None, 'user': 'hdfs', 'action': ['execute'], 'hadoop_conf_dir':
'/usr/hdp/current/hadoop-client/conf'}
>     2016-04-27 14:22:13,846 - No resources to create. 'create_on_execute' or 'delete_on_execute'
wasn't triggered before this 'execute' action.
>     2016-04-27 14:22:13,846 - Ranger admin not installed
>     
> Time 14:21:50 - 14:22:13 was spend just re-running haadmin calls, while we
> already figured
> This should give ~15s time reduce.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message