ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tristen Georgiou (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-11008) Hive Metastore Process Alert fails when Hive is running with authorization enabled
Date Thu, 07 May 2015 22:45:59 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-11008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tristen Georgiou updated AMBARI-11008:
--------------------------------------
    Description: 
Hive has been custom configured for authorization and I believe the alert is failing because
it doesn't use the configured hive username to perform it's check.

Here are the relevant config options:
hive.users.in.admin.role=hdfs
hive.security.authorization.enabled=true
hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory
hive.security.authenticator.manager=org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator=

{noformat}
Metastore on dev failed (Execution of 'export HIVE_CONF_DIR='/etc/hive/conf.server/' ; hive
--hiveconf hive.metastore.uris=thrift://dev:9083 --hiveconf hive.execution.engine=mr -e 'show
databases;' 

returned 1. 

15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.optimize.mapjoin.mapreduce does
not exist
15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.server2.enable.impersonation does
not exist
15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.auto.convert.sortmerge.join.noconditionaltask
does not exist

Logging initialized using configuration in file:/etc/hive/conf.server/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Exception in thread &quot;main&quot; java.lang.RuntimeException: java.lang.RuntimeException:
java.io.IOException: Permission denied
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: java.io.IOException: Permission denied
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:437)
	... 8 more
Caused by: java.io.IOException: Permission denied
	at java.io.UnixFileSystem.createFileExclusively(Native Method)
	at java.io.File.createNewFile(File.java:1006)
	at java.io.File.createTempFile(File.java:1989)
	at org.apache.hadoop.hive.ql.session.SessionState.createTempFile(SessionState.java:712)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:435)
	... 8 more
)" data-bindattr-3772="3772"><script id="metamorph-4680-start" type="text/x-placeholder"></script>Metastore
on dev failed (Execution of 'export HIVE_CONF_DIR='/etc/hive/conf.server/' ; hive --hiveconf
hive.metastore.uris=thrift://dev:9083 --hiveconf hive.execution.engine=mr -e 'show databases;''
returned 1. 15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.optimize.mapjoin.mapreduce
does not exist
15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.server2.enable.impersonation does
not exist
15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.auto.convert.sortmerge.join.noconditionaltask
does not exist

Logging initialized using configuration in file:/etc/hive/conf.server/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException:
Permission denied
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: java.io.IOException: Permission denied
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:437)
	... 8 more
Caused by: java.io.IOException: Permission denied
	at java.io.UnixFileSystem.createFileExclusively(Native Method)
	at java.io.File.createNewFile(File.java:1006)
	at java.io.File.createTempFile(File.java:1989)
	at org.apache.hadoop.hive.ql.session.SessionState.createTempFile(SessionState.java:712)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:435)
	... 8 more
)
{noformat}

  was:
Hive has been custom configured for authorization and I believe the alert is failing because
it doesn't use the configured hive username to perform it's check.

{noformat}
Metastore on dev failed (Execution of 'export HIVE_CONF_DIR='/etc/hive/conf.server/' ; hive
--hiveconf hive.metastore.uris=thrift://dev:9083 --hiveconf hive.execution.engine=mr -e 'show
databases;' 

returned 1. 

15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.optimize.mapjoin.mapreduce does
not exist
15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.server2.enable.impersonation does
not exist
15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.auto.convert.sortmerge.join.noconditionaltask
does not exist

Logging initialized using configuration in file:/etc/hive/conf.server/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Exception in thread &quot;main&quot; java.lang.RuntimeException: java.lang.RuntimeException:
java.io.IOException: Permission denied
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: java.io.IOException: Permission denied
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:437)
	... 8 more
Caused by: java.io.IOException: Permission denied
	at java.io.UnixFileSystem.createFileExclusively(Native Method)
	at java.io.File.createNewFile(File.java:1006)
	at java.io.File.createTempFile(File.java:1989)
	at org.apache.hadoop.hive.ql.session.SessionState.createTempFile(SessionState.java:712)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:435)
	... 8 more
)" data-bindattr-3772="3772"><script id="metamorph-4680-start" type="text/x-placeholder"></script>Metastore
on dev failed (Execution of 'export HIVE_CONF_DIR='/etc/hive/conf.server/' ; hive --hiveconf
hive.metastore.uris=thrift://dev:9083 --hiveconf hive.execution.engine=mr -e 'show databases;''
returned 1. 15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.optimize.mapjoin.mapreduce
does not exist
15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.server2.enable.impersonation does
not exist
15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.auto.convert.sortmerge.join.noconditionaltask
does not exist

Logging initialized using configuration in file:/etc/hive/conf.server/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException:
Permission denied
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: java.io.IOException: Permission denied
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:437)
	... 8 more
Caused by: java.io.IOException: Permission denied
	at java.io.UnixFileSystem.createFileExclusively(Native Method)
	at java.io.File.createNewFile(File.java:1006)
	at java.io.File.createTempFile(File.java:1989)
	at org.apache.hadoop.hive.ql.session.SessionState.createTempFile(SessionState.java:712)
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:435)
	... 8 more
)
{noformat}


> Hive Metastore Process Alert fails when Hive is running with authorization enabled
> ----------------------------------------------------------------------------------
>
>                 Key: AMBARI-11008
>                 URL: https://issues.apache.org/jira/browse/AMBARI-11008
>             Project: Ambari
>          Issue Type: Bug
>          Components: alerts
>    Affects Versions: 2.0.0
>         Environment: Ubuntu 12
>            Reporter: Tristen Georgiou
>
> Hive has been custom configured for authorization and I believe the alert is failing
because it doesn't use the configured hive username to perform it's check.
> Here are the relevant config options:
> hive.users.in.admin.role=hdfs
> hive.security.authorization.enabled=true
> hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory
> hive.security.authenticator.manager=org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator=
> {noformat}
> Metastore on dev failed (Execution of 'export HIVE_CONF_DIR='/etc/hive/conf.server/'
; hive --hiveconf hive.metastore.uris=thrift://dev:9083 --hiveconf hive.execution.engine=mr
-e 'show databases;' 
> returned 1. 
> 15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.optimize.mapjoin.mapreduce
does not exist
> 15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
> 15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.server2.enable.impersonation
does not exist
> 15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.auto.convert.sortmerge.join.noconditionaltask
does not exist
> Logging initialized using configuration in file:/etc/hive/conf.server/hive-log4j.properties
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Exception in thread &quot;main&quot; java.lang.RuntimeException: java.lang.RuntimeException:
java.io.IOException: Permission denied
> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
> 	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
> 	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> 	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.lang.RuntimeException: java.io.IOException: Permission denied
> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:437)
> 	... 8 more
> Caused by: java.io.IOException: Permission denied
> 	at java.io.UnixFileSystem.createFileExclusively(Native Method)
> 	at java.io.File.createNewFile(File.java:1006)
> 	at java.io.File.createTempFile(File.java:1989)
> 	at org.apache.hadoop.hive.ql.session.SessionState.createTempFile(SessionState.java:712)
> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:435)
> 	... 8 more
> )" data-bindattr-3772="3772"><script id="metamorph-4680-start" type="text/x-placeholder"></script>Metastore
on dev failed (Execution of 'export HIVE_CONF_DIR='/etc/hive/conf.server/' ; hive --hiveconf
hive.metastore.uris=thrift://dev:9083 --hiveconf hive.execution.engine=mr -e 'show databases;''
returned 1. 15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.optimize.mapjoin.mapreduce
does not exist
> 15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.heapsize does not exist
> 15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.server2.enable.impersonation
does not exist
> 15/05/07 22:38:46 WARN conf.HiveConf: HiveConf of name hive.auto.convert.sortmerge.join.noconditionaltask
does not exist
> Logging initialized using configuration in file:/etc/hive/conf.server/hive-log4j.properties
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hadoop/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/usr/hdp/2.2.4.2-2/hive/lib/hive-jdbc-0.14.0.2.2.4.2-2-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException:
Permission denied
> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
> 	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
> 	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> 	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.lang.RuntimeException: java.io.IOException: Permission denied
> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:437)
> 	... 8 more
> Caused by: java.io.IOException: Permission denied
> 	at java.io.UnixFileSystem.createFileExclusively(Native Method)
> 	at java.io.File.createNewFile(File.java:1006)
> 	at java.io.File.createTempFile(File.java:1989)
> 	at org.apache.hadoop.hive.ql.session.SessionState.createTempFile(SessionState.java:712)
> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:435)
> 	... 8 more
> )
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message