ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Hill <greg.h...@RACKSPACE.COM>
Subject Did something get broken for webhcat today?
Date Wed, 18 Mar 2015 16:22:27 GMT
Starting this morning, we started seeing this on every single install.  I think someone at
Hortonworks pushed out a broken RPM or something.  Any ideas?  This is rather urgent as we
are no longer able to provision HDP 2.2 clusters at all because of it.


2015-03-18 15:58:05,982 - Group['hadoop'] {'ignore_failures': False}
2015-03-18 15:58:05,984 - Modifying group hadoop
2015-03-18 15:58:06,080 - Group['nobody'] {'ignore_failures': False}
2015-03-18 15:58:06,081 - Modifying group nobody
2015-03-18 15:58:06,219 - Group['users'] {'ignore_failures': False}
2015-03-18 15:58:06,220 - Modifying group users
2015-03-18 15:58:06,370 - Group['nagios'] {'ignore_failures': False}
2015-03-18 15:58:06,371 - Modifying group nagios
2015-03-18 15:58:06,474 - User['nobody'] {'gid': 'hadoop', 'ignore_failures': False, 'groups':
[u'nobody']}
2015-03-18 15:58:06,475 - Modifying user nobody
2015-03-18 15:58:06,558 - User['hive'] {'gid': 'hadoop', 'ignore_failures': False, 'groups':
[u'hadoop']}
2015-03-18 15:58:06,559 - Modifying user hive
2015-03-18 15:58:06,634 - User['mapred'] {'gid': 'hadoop', 'ignore_failures': False, 'groups':
[u'hadoop']}
2015-03-18 15:58:06,635 - Modifying user mapred
2015-03-18 15:58:06,722 - User['nagios'] {'gid': 'nagios', 'ignore_failures': False, 'groups':
[u'hadoop']}
2015-03-18 15:58:06,723 - Modifying user nagios
2015-03-18 15:58:06,841 - User['ambari-qa'] {'gid': 'hadoop', 'ignore_failures': False, 'groups':
[u'users']}
2015-03-18 15:58:06,842 - Modifying user ambari-qa
2015-03-18 15:58:06,963 - User['zookeeper'] {'gid': 'hadoop', 'ignore_failures': False, 'groups':
[u'hadoop']}
2015-03-18 15:58:06,964 - Modifying user zookeeper
2015-03-18 15:58:07,093 - User['tez'] {'gid': 'hadoop', 'ignore_failures': False, 'groups':
[u'users']}
2015-03-18 15:58:07,094 - Modifying user tez
2015-03-18 15:58:07,217 - User['hdfs'] {'gid': 'hadoop', 'ignore_failures': False, 'groups':
[u'hadoop']}
2015-03-18 15:58:07,218 - Modifying user hdfs
2015-03-18 15:58:07,354 - User['yarn'] {'gid': 'hadoop', 'ignore_failures': False, 'groups':
[u'hadoop']}
2015-03-18 15:58:07,355 - Modifying user yarn
2015-03-18 15:58:07,485 - User['hcat'] {'gid': 'hadoop', 'ignore_failures': False, 'groups':
[u'hadoop']}
2015-03-18 15:58:07,486 - Modifying user hcat
2015-03-18 15:58:07,629 - File['/var/lib/ambari-agent/data/tmp/changeUid.sh'] {'content':
StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2015-03-18 15:58:07,631 - Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa
2>/dev/null'] {'not_if': 'test $(id -u ambari-qa) -gt 1000'}
2015-03-18 15:58:07,768 - Skipping Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh ambari-qa
/tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa
2>/dev/null'] due to not_if
2015-03-18 15:58:07,769 - Directory['/etc/hadoop/conf.empty'] {'owner': 'root', 'group': 'root',
'recursive': True}
2015-03-18 15:58:07,770 - Link['/etc/hadoop/conf'] {'not_if': 'ls /etc/hadoop/conf', 'to':
'/etc/hadoop/conf.empty'}
2015-03-18 15:58:07,895 - Skipping Link['/etc/hadoop/conf'] due to not_if
2015-03-18 15:58:07,960 - File['/etc/hadoop/conf/hadoop-env.sh'] {'content': InlineTemplate(...),
'owner': 'hdfs'}
2015-03-18 15:58:08,092 - Execute['/bin/echo 0 > /selinux/enforce'] {'only_if': 'test -f
/selinux/enforce'}
2015-03-18 15:58:08,240 - Skipping Execute['/bin/echo 0 > /selinux/enforce'] due to only_if
2015-03-18 15:58:08,241 - Directory['/var/log/hadoop'] {'owner': 'root', 'group': 'hadoop',
'mode': 0775, 'recursive': True}
2015-03-18 15:58:08,244 - Directory['/var/run/hadoop'] {'owner': 'root', 'group': 'root',
'recursive': True}
2015-03-18 15:58:08,250 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'recursive': True}
2015-03-18 15:58:08,278 - File['/etc/hadoop/conf/commons-logging.properties'] {'content':
Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2015-03-18 15:58:08,288 - File['/etc/hadoop/conf/health_check'] {'content': Template('health_check-v2.j2'),
'owner': 'hdfs'}
2015-03-18 15:58:08,295 - File['/etc/hadoop/conf/log4j.properties'] {'content': '...', 'owner':
'hdfs', 'group': 'hadoop', 'mode': 0644}
2015-03-18 15:58:08,322 - File['/etc/hadoop/conf/hadoop-metrics2.properties'] {'content':
Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs'}
2015-03-18 15:58:08,325 - File['/etc/hadoop/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'),
'mode': 0755}
2015-03-18 15:58:08,330 - File['/etc/hadoop/conf/configuration.xsl'] {'owner': 'hdfs', 'group':
'hadoop'}
2015-03-18 15:58:09,219 - HdfsDirectory['/user/hcat'] {'security_enabled': False, 'keytab':
[EMPTY], 'conf_dir': '/etc/hadoop/conf', 'hdfs_user': 'hdfs', 'kinit_path_local': '', 'mode':
0755, 'owner': 'hcat', 'bin_dir': '/usr/hdp/current/hadoop-client/bin', 'action': ['create_delayed']}
2015-03-18 15:58:09,220 - HdfsDirectory['None'] {'security_enabled': False, 'keytab': [EMPTY],
'conf_dir': '/etc/hadoop/conf', 'hdfs_user': 'hdfs', 'kinit_path_local': '', 'action': ['create'],
'bin_dir': '/usr/hdp/current/hadoop-client/bin'}
2015-03-18 15:58:09,228 - Execute['hadoop --config /etc/hadoop/conf fs -mkdir `rpm -q hadoop
| grep -q "hadoop-1" || echo "-p"` /user/hcat && hadoop --config /etc/hadoop/conf
fs -chmod  755 /user/hcat && hadoop --config /etc/hadoop/conf fs -chown  hcat /user/hcat']
{'not_if': "su - hdfs -c 'export PATH=$PATH:/usr/hdp/current/hadoop-client/bin ; hadoop --config
/etc/hadoop/conf fs -ls /user/hcat'", 'user': 'hdfs', 'path': ['/usr/hdp/current/hadoop-client/bin']}
2015-03-18 15:58:37,822 - Directory['/var/run/webhcat'] {'owner': 'hcat', 'group': 'hadoop',
'recursive': True, 'mode': 0755}
2015-03-18 15:58:37,823 - Changing group for /var/run/webhcat from 0 to hadoop
2015-03-18 15:58:37,823 - Directory['/var/log/webhcat'] {'owner': 'hcat', 'group': 'hadoop',
'recursive': True, 'mode': 0755}
2015-03-18 15:58:37,824 - Creating directory Directory['/var/log/webhcat']
2015-03-18 15:58:37,824 - Changing owner for /var/log/webhcat from 0 to hcat
2015-03-18 15:58:37,824 - Changing group for /var/log/webhcat from 0 to hadoop
2015-03-18 15:58:37,824 - Directory['/etc/hive-webhcat/conf'] {'owner': 'hcat', 'group': 'hadoop',
'recursive': True}
2015-03-18 15:58:37,825 - Changing owner for /etc/hive-webhcat/conf from 0 to hcat
2015-03-18 15:58:37,825 - Changing group for /etc/hive-webhcat/conf from 0 to hadoop
2015-03-18 15:58:37,893 - ExecuteHadoop['fs -ls hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/hive/hive.tar.gz'] {'logoutput': True, 'bin_dir': '/usr/hdp/current/hadoop-client/bin',
'user': 'hcat', 'conf_dir': '/etc/hadoop/conf'}
2015-03-18 15:58:37,896 - Execute['hadoop --config /etc/hadoop/conf fs -ls hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/hive/hive.tar.gz'] {'logoutput': True, 'path': ['/usr/hdp/current/hadoop-client/bin'],
'tries': 1, 'user': 'hcat', 'try_sleep': 0}
2015-03-18 15:58:43,597 - -bash: line 1: 2.2.3.0-2611/hive/hive.tar.gz: No such file or directory
2015-03-18 15:58:43,599 - HdfsDirectory['hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/hive'] {'security_enabled': False, 'keytab': [EMPTY], 'conf_dir': '/etc/hadoop/conf',
'hdfs_user': 'hdfs', 'kinit_path_local': '', 'mode': 0555, 'owner': 'hdfs', 'bin_dir': '/usr/hdp/current/hadoop-client/bin',
'action': ['create']}
2015-03-18 15:58:43,601 - Execute['hadoop --config /etc/hadoop/conf fs -mkdir `rpm -q hadoop
| grep -q "hadoop-1" || echo "-p"` hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/hive && hadoop --config /etc/hadoop/conf fs -chmod  555 hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/hive && hadoop --config /etc/hadoop/conf fs -chown  hdfs hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/hive'] {'not_if': "su - hdfs -c 'export PATH=$PATH:/usr/hdp/current/hadoop-client/bin
; hadoop --config /etc/hadoop/conf fs -ls hdfs:///hdp/apps/2.2.0.0-2041\n2.2.3.0-2611/hive'",
'user': 'hdfs', 'path': ['/usr/hdp/current/hadoop-client/bin']}
2015-03-18 15:58:54,904 - ExecuteHadoop['fs -ls hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/pig/pig.tar.gz'] {'logoutput': True, 'bin_dir': '/usr/hdp/current/hadoop-client/bin',
'user': 'hcat', 'conf_dir': '/etc/hadoop/conf'}
2015-03-18 15:58:54,906 - Execute['hadoop --config /etc/hadoop/conf fs -ls hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/pig/pig.tar.gz'] {'logoutput': True, 'path': ['/usr/hdp/current/hadoop-client/bin'],
'tries': 1, 'user': 'hcat', 'try_sleep': 0}
2015-03-18 15:59:00,322 - -bash: line 1: 2.2.3.0-2611/pig/pig.tar.gz: No such file or directory
2015-03-18 15:59:00,323 - HdfsDirectory['hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/pig'] {'security_enabled': False, 'keytab': [EMPTY], 'conf_dir': '/etc/hadoop/conf',
'hdfs_user': 'hdfs', 'kinit_path_local': '', 'mode': 0555, 'owner': 'hdfs', 'bin_dir': '/usr/hdp/current/hadoop-client/bin',
'action': ['create']}
2015-03-18 15:59:00,326 - Execute['hadoop --config /etc/hadoop/conf fs -mkdir `rpm -q hadoop
| grep -q "hadoop-1" || echo "-p"` hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/hive hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/pig && hadoop --config /etc/hadoop/conf fs -chmod  555 hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/hive hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/pig && hadoop --config /etc/hadoop/conf fs -chown  hdfs hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/hive hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/pig'] {'not_if': "su - hdfs -c 'export PATH=$PATH:/usr/hdp/current/hadoop-client/bin
; hadoop --config /etc/hadoop/conf fs -ls hdfs:///hdp/apps/2.2.0.0-2041\n2.2.3.0-2611/hive
hdfs:///hdp/apps/2.2.0.0-2041\n2.2.3.0-2611/pig'", 'user': 'hdfs', 'path': ['/usr/hdp/current/hadoop-client/bin']}
2015-03-18 15:59:11,576 - ExecuteHadoop['fs -ls hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/mapreduce/hadoop-streaming.jar'] {'logoutput': True, 'bin_dir': '/usr/hdp/current/hadoop-client/bin',
'user': 'hcat', 'conf_dir': '/etc/hadoop/conf'}
2015-03-18 15:59:11,578 - Execute['hadoop --config /etc/hadoop/conf fs -ls hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/mapreduce/hadoop-streaming.jar'] {'logoutput': True, 'path': ['/usr/hdp/current/hadoop-client/bin'],
'tries': 1, 'user': 'hcat', 'try_sleep': 0}
2015-03-18 15:59:17,094 - -bash: line 1: 2.2.3.0-2611/mapreduce/hadoop-streaming.jar: No such
file or directory
2015-03-18 15:59:17,097 - HdfsDirectory['hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/mapreduce'] {'security_enabled': False, 'keytab': [EMPTY], 'conf_dir': '/etc/hadoop/conf',
'hdfs_user': 'hdfs', 'kinit_path_local': '', 'mode': 0555, 'owner': 'hdfs', 'bin_dir': '/usr/hdp/current/hadoop-client/bin',
'action': ['create']}
2015-03-18 15:59:17,099 - Execute['hadoop --config /etc/hadoop/conf fs -mkdir `rpm -q hadoop
| grep -q "hadoop-1" || echo "-p"` hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/hive hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/pig hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/mapreduce && hadoop --config /etc/hadoop/conf fs -chmod  555 hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/hive hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/pig hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/mapreduce && hadoop --config /etc/hadoop/conf fs -chown  hdfs hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/hive hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/pig hdfs:///hdp/apps/2.2.0.0-2041
2.2.3.0-2611/mapreduce'] {'not_if': '...', 'user': 'hdfs', 'path': ['/usr/hdp/current/hadoop-client/bin']}
2015-03-18 15:59:28,070 - Could not find file: /usr/hdp/current/sqoop-client/sqoop.tar.gz
2015-03-18 15:59:28,071 - XmlConfig['webhcat-site.xml'] {'owner': 'hcat', 'group': 'hadoop',
'conf_dir': '/etc/hive-webhcat/conf', 'configuration_attributes': ..., 'configurations': ...}
2015-03-18 15:59:28,090 - Generating config: /etc/hive-webhcat/conf/webhcat-site.xml
2015-03-18 15:59:28,091 - File['/etc/hive-webhcat/conf/webhcat-site.xml'] {'owner': 'hcat',
'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding': 'UTF-8'}
2015-03-18 15:59:28,092 - Writing File['/etc/hive-webhcat/conf/webhcat-site.xml'] because
it doesn't exist
2015-03-18 15:59:28,093 - Changing owner for /etc/hive-webhcat/conf/webhcat-site.xml from
0 to hcat
2015-03-18 15:59:28,093 - Changing group for /etc/hive-webhcat/conf/webhcat-site.xml from
0 to hadoop
2015-03-18 15:59:28,095 - File['/etc/hive-webhcat/conf/webhcat-env.sh'] {'content': InlineTemplate(...),
'owner': 'hcat', 'group': 'hadoop'}
2015-03-18 15:59:28,096 - Writing File['/etc/hive-webhcat/conf/webhcat-env.sh'] because it
doesn't exist
2015-03-18 15:59:28,096 - Changing owner for /etc/hive-webhcat/conf/webhcat-env.sh from 0
to hcat
2015-03-18 15:59:28,096 - Changing group for /etc/hive-webhcat/conf/webhcat-env.sh from 0
to hadoop
2015-03-18 15:59:28,097 - Execute['env HADOOP_HOME=/usr/hdp/current/hadoop-client /usr/hdp/current/hive-webhcat/sbin/webhcat_server.sh
start'] {'not_if': 'ls /var/run/webhcat/webhcat.pid >/dev/null 2>&1 && ps
`cat /var/run/webhcat/webhcat.pid` >/dev/null 2>&1', 'user': 'hcat'}
2015-03-18 15:59:28,179 - Error while executing command 'start':
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 123, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/webhcat_server.py",
line 39, in start
    webhcat_service(action = 'start')
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/webhcat_service.py",
line 33, in webhcat_service
    not_if=no_op_test
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149,
in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115,
in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line
241, in action_run
    raise ex
Fail: Execution of 'env HADOOP_HOME=/usr/hdp/current/hadoop-client /usr/hdp/current/hive-webhcat/sbin/webhcat_server.sh
start' returned 127. env: /usr/hdp/current/hive-webhcat/sbin/webhcat_server.sh: No such file
or directory

Mime
View raw message