ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitro Lisnichenko" <dlysniche...@hortonworks.com>
Subject Re: Review Request 31414: [Monarch] Cluster create failed with timeout on the client side
Date Wed, 25 Feb 2015 16:27:43 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31414/#review74032
-----------------------------------------------------------

Ship it!


Ship It!

- Dmitro Lisnichenko


On Feb. 25, 2015, 3:45 p.m., Andrew Onischuk wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31414/
> -----------------------------------------------------------
> 
> (Updated Feb. 25, 2015, 3:45 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-9795
>     https://issues.apache.org/jira/browse/AMBARI-9795
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> We saw one cluster create fail in Production with the below error during
> Ambari install:
> 
>     
>     
>     
>     Fail: Execution of 'apt-get update -qq -o Dir::Etc::sourcelist=sources.list.d/HDP.list
-o APT::Get::List-Cleanup=0' returned 100. E: Could not get lock /var/lib/apt/lists/lock -
open (11: Resource temporarily unavailable)\nE: Unable to lock directory /var/lib/apt/lists/"
>     
> 
> Detailed log:
> 
>     
>     
>     
>         {
>           "href" : "http://headnode0.mjlinux15-ssh.c3.internal.cloudapp.net:8080/api/v1/clusters/mjlinux15/requests/1/tasks/27",
>           "Tasks" : {
>             "cluster_name" : "mjlinux15",
>             "command_detail" : "DATANODE INSTALL",
>             "id" : 27,
>             "request_id" : 1,
>             "stage_id" : 1,
>             "status" : "FAILED",
>             "stderr" : "2015-02-18 06:55:16,758 - Error while executing command 'install':\nTraceback
(most recent call last):\n  File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\",
line 184, in execute\n    method(env)\n  File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py\",
line 33, in hook\n    install_repos()\n  File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py\",
line 55, in install_repos\n    _alter_repo(\"create\", params.repo_info, template)\n  File
\"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py\",
line 49, in _alter_repo\n    components = ubuntu_components, # ubuntu specific\n  File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\",
line 148, in __init__\n    self.env.run()\n  File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\",
line 151, in run
 \n    self.run_action(resource, action)\n  File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\",
line 117, in run_action\n    provider_action()\n  File \"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/repository.py\",
line 97, in action_create\n    retcode, out = checked_call(update_cmd_formatted, sudo=True)\n
 File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 67, in
inner\n    return function(command, **kwargs)\n  File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\",
line 79, in checked_call\n    return _call(command, logoutput, True, cwd, env, preexec_fn,
user, wait_for_finish, timeout, path, sudo, on_new_line)\n  File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\",
line 185, in _call\n    raise Fail(err_msg)\nFail: Execution of 'apt-get update -qq -o Dir::Etc::sourcelist=sources.list.d/HDP.list
-o APT::Get::List-Cleanup=0' returned 100. E: Could no
 t get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)\nE: Unable
to lock directory /var/lib/apt/lists/"
>           }
>         },
>         {
>           "href" : "http://headnode0.mjlinux15-ssh.c3.internal.cloudapp.net:8080/api/v1/clusters/mjlinux15/requests/1/tasks/28",
>           "Tasks" : {
>             "cluster_name" : "mjlinux15",
>             "command_detail" : "METRIC_MONITOR INSTALL",
>             "id" : 28,
>             "request_id" : 1,
>             "stage_id" : 1,
>             "status" : "FAILED",
>             "stderr" : "2015-02-18 06:55:18,118 - Error while executing command 'install':\nTraceback
(most recent call last):\n  File \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\",
line 184, in execute\n    method(env)\n  File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py\",
line 33, in hook\n    install_repos()\n  File \"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py\",
line 55, in install_repos\n    _alter_repo(\"create\", params.repo_info, template)\n  File
\"/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py\",
line 49, in _alter_repo\n    components = ubuntu_components, # ubuntu specific\n  File \"/usr/lib/python2.6/site-packages/resource_management/core/base.py\",
line 148, in __init__\n    self.env.run()\n  File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\",
line 151, in run
 \n    self.run_action(resource, action)\n  File \"/usr/lib/python2.6/site-packages/resource_management/core/environment.py\",
line 117, in run_action\n    provider_action()\n  File \"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/repository.py\",
line 97, in action_create\n    retcode, out = checked_call(update_cmd_formatted, sudo=True)\n
 File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 67, in
inner\n    return function(command, **kwargs)\n  File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\",
line 79, in checked_call\n    return _call(command, logoutput, True, cwd, env, preexec_fn,
user, wait_for_finish, timeout, path, sudo, on_new_line)\n  File \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\",
line 185, in _call\n    raise Fail(err_msg)\nFail: Execution of 'apt-get update -qq -o Dir::Etc::sourcelist=sources.list.d/HDP-UTILS.list
-o APT::Get::List-Cleanup=0' returned 100. E: Co
 uld not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)\nE:
Unable to lock directory /var/lib/apt/lists/"
>           }
>         },
>     
> 
> This is being seen on Azure with Ubuntu and we need to find out why this is
> happening and see if this is an OS issue or Ambari issue.
> 
> 
> Diffs
> -----
> 
>   ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py bf24730 
>   ambari-agent/src/main/python/ambari_agent/HostCleanup.py 398502e 
>   ambari-agent/src/main/python/ambari_agent/HostInfo.py dc0834a 
>   ambari-agent/src/test/python/ambari_agent/TestHostCheckReportFileHandler.py eb7efb2

>   ambari-agent/src/test/python/ambari_agent/TestHostCleanup.py f43784c 
>   ambari-agent/src/test/python/ambari_agent/TestHostInfo.py c1b93d6 
>   ambari-common/src/main/python/resource_management/libraries/functions/packages_analyzer.py
888387c 
>   ambari-server/src/main/resources/custom_actions/scripts/check_host.py ff2eabd 
>   ambari-server/src/test/python/custom_actions/TestCheckHost.py 105a0b0 
> 
> Diff: https://reviews.apache.org/r/31414/diff/
> 
> 
> Testing
> -------
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message