ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pratik Gadiya <pratik_gad...@persistent.com>
Subject RE: Hive Metastore Service Startup Fails
Date Wed, 03 Jun 2015 10:01:53 GMT
Thanks Yusako

I think it's better for me to remove the special characters from the password strings while
forming the blueprint.

I hope that will result into a stable deployment of the clusters ahead :)

~Pratik


From: Yusaku Sako [mailto:yusaku@hortonworks.com]
Sent: Wednesday, June 03, 2015 2:46 PM
To: user@ambari.apache.org
Subject: Re: Hive Metastore Service Startup Fails

Hi Pratik,

In the Blueprint, try using a password without the ampersand "&" character.
I believe it's not working because Ambari is executing the following command:

export HIVE_CONF_DIR=/etc/hive/conf.server ; /usr/hdp/current/hive-client/bin/schematool -initSchema
-dbType mysql -userName hive -passWord tkdw1rN&

& has a special meaning in the shell.
So essentially, Ambari invoking it like this and using an incorrect password:

export HIVE_CONF_DIR=/etc/hive/conf.server ; /usr/hdp/current/hive-client/bin/schematool -initSchema
-dbType mysql -userName hive -passWord tkdw1rN &

Ambari should be escaping such special characters, but it doesn't look like it's escaping
properly.

I hope this helps.
Yusaku

From: Pratik Gadiya <pratik_gadiya@persistent.com<mailto:pratik_gadiya@persistent.com>>
Reply-To: "user@ambari.apache.org<mailto:user@ambari.apache.org>" <user@ambari.apache.org<mailto:user@ambari.apache.org>>
Date: Wednesday, June 3, 2015 12:48 AM
To: "user@ambari.apache.org<mailto:user@ambari.apache.org>" <user@ambari.apache.org<mailto:user@ambari.apache.org>>
Subject: RE: Hive Metastore Service Startup Fails

Any Help Appreciated !!

From: Pratik Gadiya [mailto:pratik_gadiya@persistent.com]
Sent: Monday, June 01, 2015 5:21 PM
To: user@ambari.apache.org<mailto:user@ambari.apache.org>
Subject: FW: Hive Metastore Service Startup Fails
Importance: High

Hello All,

When I try to deploy hortonworks cluster using ambari blueprint APIs, it results in failure
while starting up of Hive Metastore service.

The same blueprint most of the times works appropriately on the same environment.

The parameter which gets changed in the entire blueprint w.r.t hive is,

Host Mapping File Content:
{'blueprint': 'onemasterblueprint',
'configurations': [{u'hive-env': {u'hive_metastore_user_passwd': 'tkdw1rN&'}},
                    {u'gateway-site': {u'gateway.port': u'8445'}},
                    {u'nagios-env': {u'nagios_contact': u'abc@us.ibm.com<mailto:u'abc@us.ibm.com>'}},
                    {u'hive-site': {u'javax.jdo.option.ConnectionPassword': 'tkdw1rN&'}},
                    {'hdfs-site': {'dfs.datanode.data.dir': '/disk1/hadoop/hdfs/data,/disk2/hadoop/hdfs/data',
                                   'dfs.namenode.checkpoint.dir': '/disk1/hadoop/hdfs/namesecondary',
                                   'dfs.namenode.name.dir': '/disk1/hadoop/hdfs/namenode'}},
                    {'core-site': {'fs.swift.impl': 'org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem',
                                   'fs.swift.service.softlayer.auth.url': 'https://dal05.objectstorage.service.networklayer.com/auth/v1.0',
                                   'fs.swift.service.softlayer.connect.timeout': '120000',
                                   'fs.swift.service.softlayer.public': 'false',
                                   'fs.swift.service.softlayer.use.encryption': 'true',
                                   'fs.swift.service.softlayer.use.get.auth': 'true'}}],
'default_password': 'tkdw1rN&',
'host_groups': [{'hosts': [{'fqdn': 'vmktest0003.test.analytics.com'}],
                  'name': 'master'},
                 {'hosts': [{'fqdn': 'vmktest0004.test.analytics.com'}],
                  'name': 'compute'}]}

Error.txt:
2015-06-01 05:59:22,178 - Error while executing command 'start':
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 123, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive_metastore.py",
line 43, in start
    self.configure(env) # FOR SECURITY
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive_metastore.py",
line 38, in configure
    hive(name='metastore')
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive.py",
line 97, in hive
    not_if = check_schema_created_cmd
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149,
in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115,
in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line
241, in action_run
    raise ex
Fail: Execution of 'export HIVE_CONF_DIR=/etc/hive/conf.server ; /usr/hdp/current/hive-client/bin/schematool
-initSchema -dbType mysql -userName hive -passWord [PROTECTED]' returned 1. 15/06/01 05:59:21
WARN conf.HiveConf: HiveConf of name hive.optimize.mapjoin.mapreduce does not exist
15/06/01 05:59:21 WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
15/06/01 05:59:21 WARN conf.HiveConf: HiveConf of name hive.server2.enable.impersonation does
not exist
15/06/01 05:59:21 WARN conf.HiveConf: HiveConf of name hive.auto.convert.sortmerge.join.noconditionaltask
does not exist
Metastore connection URL: jdbc:mysql://vmktest0009.test.analytics.ibmcloud.com/hive?createDatabaseIfNotExist=true
Metastore Connection Driver :      com.mysql.jdbc.Driver
Metastore connection User:         hive
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
*** schemaTool failed ***

Output.txt:


2015-06-01 05:59:07,907 - Changing permission for /var/lib/ambari-agent/data/tmp/start_metastore_script
from 644 to 755

2015-06-01 05:59:07,909 - Execute['export HIVE_CONF_DIR=/etc/hive/conf.server ; /usr/hdp/current/hive-client/bin/schematool
-initSchema -dbType mysql -userName hive -passWord [PROTECTED]'] {'not_if': 'export HIVE_CONF_DIR=/etc/hive/conf.server
; /usr/hdp/current/hive-client/bin/schematool -info -dbType mysql -userName hive -passWord
\'Hb2\'"\'"\'aasz\''}

2015-06-01 05:59:22,178 - Error while executing command 'start':

Traceback (most recent call last):

  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 123, in execute

    method(env)

  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive_metastore.py",
line 43, in start

    self.configure(env) # FOR SECURITY

  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive_metastore.py",
line 38, in configure

    hive(name='metastore')

  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive.py",
line 97, in hive

    not_if = check_schema_created_cmd

  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__

    self.env.run()

  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149,
in run

    self.run_action(resource, action)

  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115,
in run_action

    provider_action()

  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line
241, in action_run

    raise ex

Fail: Execution of 'export HIVE_CONF_DIR=/etc/hive/conf.server ; /usr/hdp/current/hive-client/bin/schematool
-initSchema -dbType mysql -userName hive -passWord [PROTECTED]' returned 1. 15/06/01 05:59:21
WARN conf.HiveConf: HiveConf of name hive.optimize.mapjoin.mapreduce does not exist

15/06/01 05:59:21 WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist

15/06/01 05:59:21 WARN conf.HiveConf: HiveConf of name hive.server2.enable.impersonation does
not exist

15/06/01 05:59:21 WARN conf.HiveConf: HiveConf of name hive.auto.convert.sortmerge.join.noconditionaltask
does not exist

Metastore connection URL:  jdbc:mysql://vmktest0009.test.analytics.ibmcloud.com/hive?createDatabaseIfNotExist=true

Metastore Connection Driver :       com.mysql.jdbc.Driver

Metastore connection User:          hive

org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.

*** schemaTool failed ***


Is there any constraint w.r.t setting up of passwords in ambari.

Please let me know how can I resolve this error so that I can automate the same in the deployment.


With Regards,
Pratik Gadiya


DISCLAIMER ========== This e-mail may contain privileged and confidential information which
is the property of Persistent Systems Ltd. It is intended only for the use of the individual
or entity to which it is addressed. If you are not the intended recipient, you are not authorized
to read, retain, copy, print, distribute or use this message. If you have received this communication
in error, please notify the sender and delete all copies of this message. Persistent Systems
Ltd. does not accept any liability for virus infected mails.

DISCLAIMER ========== This e-mail may contain privileged and confidential information which
is the property of Persistent Systems Ltd. It is intended only for the use of the individual
or entity to which it is addressed. If you are not the intended recipient, you are not authorized
to read, retain, copy, print, distribute or use this message. If you have received this communication
in error, please notify the sender and delete all copies of this message. Persistent Systems
Ltd. does not accept any liability for virus infected mails.

DISCLAIMER ========== This e-mail may contain privileged and confidential information which
is the property of Persistent Systems Ltd. It is intended only for the use of the individual
or entity to which it is addressed. If you are not the intended recipient, you are not authorized
to read, retain, copy, print, distribute or use this message. If you have received this communication
in error, please notify the sender and delete all copies of this message. Persistent Systems
Ltd. does not accept any liability for virus infected mails.

DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent
Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed.
If you are not the intended recipient, you are not authorized to read, retain, copy, print,
distribute or use this message. If you have received this communication in error, please notify
the sender and delete all copies of this message. Persistent Systems Ltd. does not accept
any liability for virus infected mails.


Mime
View raw message