incubator-ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sumit Mohanty <smoha...@hortonworks.com>
Subject Re: Problem installing Datanode
Date Wed, 18 Sep 2013 16:00:16 GMT
At this point Ambari does not have the ability to configure the timeout.
We will open a JIRA for that.

-Sumit

On 9/18/13 7:55 AM, "Jon Maron" <jmaron@hortonworks.com> wrote:

>
>On Sep 18, 2013, at 10:47 AM, Sumit Mohanty <smohanty@hortonworks.com>
>wrote:
>
>> Jon,
>> 
>> A task can timeout after 10 minutes.
>
>Is there a mechanism to increase the timeout?  Apparently connectivity is
>an issue for the given QA lab.
>
>> Datanode is typically the first
>> component to be installed. So it brings in the most number of new
>>packages.
>> We have noticed such timeouts if the environment has slow connectivity.
>>I
>> had investigated a similar situation with John few months back.
>> 
>> Typically, retry will fix it.
>> 
>> If it helps, you can setup a local repo where the installation does not
>> have to download packages from S3.
>
>We do have a VM image with pre-installed elements.  We will direct them
>to try that one as well.
>
>> What version of HDP and Ambari are you
>> using?
>> 
>> -Sumit
>> 
>> On 9/18/13 7:34 AM, "Jon Maron" <jmaron@hortonworks.com> wrote:
>> 
>>> Hi,
>>> 
>>> We are working with the savanna QA folks, and during attempt to create
>>> a cluster or scale a cluster they are seeing failures during service
>>> installation.  Below is an example of a DATANODE related failure.  It
>>> appears that the installation may be proceeding well but simply times
>>> out?  Is there any additional diagnoses we can do to ascertain the
>>>issue
>>> trigger the failure?
>>> 
>>> Here is the JSON output for the failure:
>>> 
>>> {
>>> "href" : 
>>> 
>>>"http://172.18.168.48:8080/api/v1/clusters/test-cluster/requests/1/tasks
>>>/7
>>> ",
>>> "Tasks" : {
>>>   "attempt_cnt" : 1,
>>>   "cluster_name" : "test-cluster",
>>>   "command" : "INSTALL",
>>>   "exit_code" : 999,
>>>   "host_name" : "test-cluster-worker-node-001.novalocal",
>>>   "id" : 7,
>>>   "request_id" : 1,
>>>   "role" : "DATANODE",
>>>   "stage_id" : 1,
>>>   "start_time" : 1379513216345,
>>>   "status" : "FAILED",
>>>   "stderr" : "none\n\n Puppet has been killed due to timeout",
>>>   "stdout" :notice:
>>> /Stage[main]/Hdp-repos::Process_repo/File[HDP]/ensure: defined content
>>>as
>>> '{md5}af147c3d39b51af76b25d4f95c69d1e9'
>>> notice: Finished catalog run in 0.04 seconds
>>> warning: Dynamic lookup of $service_state at
>>> /var/lib/ambari-agent/puppet/modules/hdp-hadoop/manifests/service.pp:76
>>> is deprecated.  Support will be removed in Puppet 2.8.  Use a
>>> fully-qualified variable name (e.g., $classname::variable) or
>>> parameterized classes.
>>> warning: Dynamic lookup of $service_state at
>>> /var/lib/ambari-agent/puppet/modules/hdp-hadoop/manifests/service.pp:85
>>> is deprecated.  Support will be removed in Puppet 2.8.  Use a
>>> fully-qualified variable name (e.g., $classname::variable) or
>>> parameterized classes.
>>> warning: Dynamic lookup of $configuration is deprecated.  Support will
>>>be
>>> removed in Puppet 2.8.  Use a fully-qualified variable name (e.g.,
>>> $classname::variable) or parameterized classes.
>>> warning: Dynamic lookup of $mapred-site is deprecated.  Support will be
>>> removed in Puppet 2.8.  Use a fully-qualified variable name (e.g.,
>>> $classname::variable) or parameterized classes.
>>> warning: Dynamic lookup of $tasktracker_port is deprecated.  Support
>>>will
>>> be removed in Puppet 2.8.  Use a fully-qualified variable name (e.g.,
>>> $classname::variable) or parameterized classes.
>>> warning: Dynamic lookup of $ambari_db_rca_url is deprecated.  Support
>>> will be removed in Puppet 2.8.  Use a fully-qualified variable name
>>> (e.g., $classname::variable) or parameterized classes.
>>> warning: Dynamic lookup of $ambari_db_rca_driver is deprecated.
>>>Support
>>> will be removed in Puppet 2.8.  Use a fully-qualified variable name
>>> (e.g., $classname::variable) or parameterized classes.
>>> warning: Dynamic lookup of $ambari_db_rca_username is deprecated.
>>> Support will be removed in Puppet 2.8.  Use a fully-qualified variable
>>> name (e.g., $classname::variable) or parameterized classes.
>>> warning: Dynamic lookup of $ambari_db_rca_password is deprecated.
>>> Support will be removed in Puppet 2.8.  Use a fully-qualified variable
>>> name (e.g., $classname::variable) or parameterized classes.
>>> notice: /Stage[1]/Hdp::Iptables/Service[iptables]/ensure: ensure
>>>changed
>>> 'running' to 'stopped'
>>> notice: 
>>>/Stage[1]/Hdp::Create_smoke_user/File[/tmp/changeUid.sh]/ensure:
>>> defined content as '{md5}8118fe9ec0bca3d841d470ad02696adf'
>>> notice: 
>>> 
>>>/Stage[1]/Hdp::Snappy::Package/Hdp::Package[snappy]/Hdp::Package::Proces
>>>s_
>>> pkg[snappy]/Package[snappy]/ensure: created
>>> notice: 
>>> /Stage[1]/Hdp/Hdp::Group[nagios_group]/Group[nagios_group]/ensure:
>>>created
>>> notice: /Stage[1]/Hdp/Hdp::User[nagios_user]/User[nagios]/ensure:
>>>created
>>> notice: 
>>> 
>>>/Stage[1]/Hdp::Snmp/Hdp::Package[snmp]/Hdp::Package::Process_pkg[snmp]/P
>>>ac
>>> kage[net-snmp-utils]/ensure: created
>>> notice: 
>>> 
>>>/Stage[1]/Hdp::Snmp/Hdp::Package[snmp]/Hdp::Package::Process_pkg[snmp]/P
>>>ac
>>> kage[net-snmp]/ensure: created
>>> notice: 
>>> 
>>>/Stage[1]/Hdp::Snmp/Hdp::Package[snmp]/Hdp::Package::Process_pkg[snmp]/H
>>>dp
>>> ::Java::Package[snmp]/Exec[mkdir -p /tmp/HDP-artifacts/ ; curl -kf
>>> --retry 10 
>>> 
>>>http://test-cluster-master-node-001.novalocal:8080/resources//jdk-6u31-l
>>>in
>>> ux-x64.bin -o /tmp/HDP-artifacts//jdk-6u31-linux-x64.bin snmp]/returns:
>>> executed successfully
>>> notice: 
>>> 
>>>/Stage[1]/Hdp::Snmp/Hdp::Package[snmp]/Hdp::Package::Process_pkg[snmp]/H
>>>dp
>>> ::Java::Package[snmp]/Exec[mkdir -p /usr/jdk64 ; chmod +x
>>> /tmp/HDP-artifacts//jdk-6u31-linux-x64.bin; cd /usr/jdk64 ; echo A |
>>> /tmp/HDP-artifacts//jdk-6u31-linux-x64.bin -noregister > /dev/null 2>&1
>>> snmp]/returns: executed successfully
>>> notice: 
>>> 
>>>/Stage[1]/Hdp::Snmp/Hdp::Package[snmp]/Hdp::Package::Process_pkg[snmp]/H
>>>dp
>>> ::Java::Package[snmp]/File[/usr/jdk64/jdk1.6.0_31/bin/java
>>>snmp]/ensure:
>>> created
>>> notice: 
>>> 
>>>/Stage[1]/Hdp::Snmp/Hdp::Snmp-configfile[snmpd.conf]/Hdp::Configfile[/et
>>>c/
>>> snmp//snmpd.conf]/File[/etc/snmp//snmpd.conf]/content: content changed
>>> '{md5}8307434bc8ed4e2a7df4928fb4232778' to
>>> '{md5}f786955c0c36f7f5a4f375e3fe93c959'
>>> notice: /Stage[1]/Hdp::Snmp/Service[snmpd]/ensure: ensure changed
>>> 'stopped' to 'running'
>>> notice: /Stage[1]/Hdp::Snmp/Service[snmpd]: Triggered 'refresh' from 1
>>> events
>>> notice: /Stage[1]/Hdp::Set_selinux/Hdp::Exec[/bin/echo 0 >
>>> /selinux/enforce]/Exec[/bin/echo 0 > /selinux/enforce]/returns:
>>>executed
>>> successfully
>>> notice: 
>>> 
>>>/Stage[1]/Hdp::Snappy::Package/Hdp::Package[snappy]/Hdp::Package::Proces
>>>s_
>>> pkg[snappy]/Package[snappy-devel]/ensure: created
>>> notice: 
>>> 
>>>/Stage[1]/Hdp::Snappy::Package/Hdp::Snappy::Package::Ln[64]/Hdp::Exec[hd
>>>p:
>>> :snappy::package::ln 64]/Exec[hdp::snappy::package::ln 64]/returns:
>>> executed successfully
>>> notice: 
>>> 
>>>/Stage[1]/Hdp::Snappy::Package/Hdp::Snappy::Package::Ln[32]/Hdp::Exec[hd
>>>p:
>>> :snappy::package::ln 32]/Exec[hdp::snappy::package::ln 32]/returns:
>>> executed successfully
>>> notice: 
>>> 
>>>/Stage[1]/Hdp/Hdp::Package[glibc]/Hdp::Package::Process_pkg[glibc]/Packa
>>>ge
>>> [glibc.i686]/ensure: created
>>> notice: 
>>> /Stage[1]/Hdp/Hdp::Group[hdp_user_group]/Group[hdp_user_group]/ensure:
>>> created
>>> notice: 
>>> 
>>>/Stage[1]/Hdp::Create_smoke_user/Hdp::User[smoke_user]/User[ambari_qa]/e
>>>ns
>>> ure: created
>>> notice: /Stage[1]/Hdp::Create_smoke_user/Hdp::Exec[/tmp/changeUid.sh
>>> ambari_qa 1012 
>>> 
>>>/tmp/hadoop-ambari_qa,/tmp/hsperfdata_ambari_qa,/home/ambari_qa,/tmp/amb
>>>ar
>>> i_qa,/tmp/sqoop-ambari_qa 2>/dev/null]/Exec[/tmp/changeUid.sh ambari_qa
>>> 1012 
>>> 
>>>/tmp/hadoop-ambari_qa,/tmp/hsperfdata_ambari_qa,/home/ambari_qa,/tmp/amb
>>>ar
>>> i_qa,/tmp/sqoop-ambari_qa 2>/dev/null]/returns: executed successfully
>>> err: 
>>> /Stage[main]/Hdp-hadoop/Hdp-hadoop::Package[hadoop]/Hdp::Package[hadoop
>>> 64]/Hdp::Package::Process_pkg[hadoop 64]/Package[hadoop-sbin]: Could
>>>not
>>> evaluate: Puppet::Util::Log requires a message
>>> 
>>> Any help would be appreciated
>>> 
>>> -- Jon
>>> 
>>> 
>>> -- 
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or
>>>entity
>>> to 
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the
>>>reader
>>> of this message is not the intended recipient, you are hereby notified
>>> that 
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>>> immediately 
>>> and delete it from your system. Thank You.
>> 
>> 
>> 
>> -- 
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or
>>entity to 
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the
>>reader 
>> of this message is not the intended recipient, you are hereby notified
>>that 
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>>immediately 
>> and delete it from your system. Thank You.
>
>
>-- 
>CONFIDENTIALITY NOTICE
>NOTICE: This message is intended for the use of the individual or entity
>to 
>which it is addressed and may contain information that is confidential,
>privileged and exempt from disclosure under applicable law. If the reader
>of this message is not the intended recipient, you are hereby notified
>that 
>any printing, copying, dissemination, distribution, disclosure or
>forwarding of this communication is strictly prohibited. If you have
>received this communication in error, please contact the sender
>immediately 
>and delete it from your system. Thank You.



-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message