hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eric wong <win19...@gmail.com>
Subject Re: User is not allowed to impersonate
Date Tue, 30 May 2017 11:25:05 GMT
you should add *hadoop.proxyuser.hue.groups=** in *hdfs-site.xml of
Namenode *

2017-05-04 21:42 GMT+08:00 Markovich <amrivkin@gmail.com>:

> i'm still unable to resolve this...
>
>  INFO  [Thread-17]: thrift.ThriftCLIService (ThriftHttpCLIService.java:run(152))
> - Started ThriftHttpCLIService in http mode on port 10001
> path=/cliservice/* with 5...500 worker threads
> 2017-05-04 13:40:14,195 INFO  [HiveServer2-HttpHandler-Pool: Thread-60]:
> thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(145)) - Could not
> validate cookie sent, will try to generate a new cookie
> 2017-05-04 13:40:14,198 INFO  [HiveServer2-HttpHandler-Pool: Thread-60]:
> thrift.ThriftHttpServlet (ThriftHttpServlet.java:doKerberosAuth(398)) -
> Failed to authenticate with http/_HOST kerberos principal, trying with
> hive/_HOST kerberos principal
> 2017-05-04 13:40:14,199 ERROR [HiveServer2-HttpHandler-Pool: Thread-60]:
> thrift.ThriftHttpServlet (ThriftHttpServlet.java:doKerberosAuth(406)) -
> Failed to authenticate with hive/_HOST kerberos principal
> 2017-05-04 13:40:14,199 ERROR [HiveServer2-HttpHandler-Pool: Thread-60]:
> thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(209)) - Error:
> org.apache.hive.service.auth.HttpAuthenticationException:
> java.lang.reflect.UndeclaredThrowableException
>         at org.apache.hive.service.cli.thrift.ThriftHttpServlet.
> doKerberosAuth(ThriftHttpServlet.java:407)
>         at org.apache.hive.service.cli.thrift.ThriftHttpServlet.
> doPost(ThriftHttpServlet.java:159)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>         at org.eclipse.jetty.servlet.ServletHolder.handle(
> ServletHolder.java:565)
>         at org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:479)
>         at org.eclipse.jetty.server.session.SessionHandler.
> doHandle(SessionHandler.java:225)
>         at org.eclipse.jetty.server.handler.ContextHandler.
> doHandle(ContextHandler.java:1031)
>         at org.eclipse.jetty.servlet.ServletHandler.doScope(
> ServletHandler.java:406)
>         at org.eclipse.jetty.server.session.SessionHandler.
> doScope(SessionHandler.java:186)
>         at org.eclipse.jetty.server.handler.ContextHandler.
> doScope(ContextHandler.java:965)
>         at org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:117)
>         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:111)
>         at org.eclipse.jetty.server.Server.handle(Server.java:349)
>         at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(
> AbstractHttpConnection.java:449)
>         at org.eclipse.jetty.server.AbstractHttpConnection$
> RequestHandler.content(AbstractHttpConnection.java:925)
>         at org.eclipse.jetty.http.HttpParser.parseNext(
> HttpParser.java:857)
>         at org.eclipse.jetty.http.HttpParser.parseAvailable(
> HttpParser.java:235)
>         at org.eclipse.jetty.server.AsyncHttpConnection.handle(
> AsyncHttpConnection.java:76)
>         at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(
> SelectChannelEndPoint.java:609)
>         at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(
> SelectChannelEndPoint.java:45)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.reflect.UndeclaredThrowableException
>         at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1742)
>         at org.apache.hive.service.cli.thrift.ThriftHttpServlet.
> doKerberosAuth(ThriftHttpServlet.java:404)
>         ... 23 more
> Caused by: org.apache.hive.service.auth.HttpAuthenticationException:
> Authorization header received from the client is empty.
>         at org.apache.hive.service.cli.thrift.ThriftHttpServlet.
> getAuthHeader(ThriftHttpServlet.java:548)
>         at org.apache.hive.service.cli.thrift.ThriftHttpServlet.
> access$100(ThriftHttpServlet.java:74)
>         at org.apache.hive.service.cli.thrift.ThriftHttpServlet$
> HttpKerberosServerAction.run(ThriftHttpServlet.java:449)
>         at org.apache.hive.service.cli.thrift.ThriftHttpServlet$
> HttpKerberosServerAction.run(ThriftHttpServlet.java:412)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(
> UserGroupInformation.java:1724)
>         ... 24 more
> 2017-05-04 13:40:14,211 INFO  [HiveServer2-HttpHandler-Pool: Thread-60]:
> thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(145)) - Could not
> validate cookie sent, will try to generate a new cookie
> 2017-05-04 13:40:14,219 INFO  [HiveServer2-HttpHandler-Pool: Thread-60]:
> thrift.ThriftHttpServlet (ThriftHttpServlet.java:doPost(204)) - Cookie
> added for clientUserName hue
> 2017-05-04 13:40:14,229 INFO  [HiveServer2-HttpHandler-Pool: Thread-60]:
> thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(313)) - Client
> protocol version: HIVE_CLI_SERVICE_PROTOCOL_V7
> 2017-05-04 13:40:14,244 WARN  [HiveServer2-HttpHandler-Pool: Thread-60]:
> thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(327)) - Error
> opening session:
> org.apache.hive.service.cli.HiveSQLException: Failed to validate proxy
> privilege of hue for hdfs
>         at org.apache.hive.service.auth.HiveAuthFactory.verifyProxyAccess(
> HiveAuthFactory.java:396)
>         at org.apache.hive.service.cli.thrift.ThriftCLIService.
> getProxyUser(ThriftCLIService.java:751)
>         at org.apache.hive.service.cli.thrift.ThriftCLIService.
> getUserName(ThriftCLIService.java:386)
>         at org.apache.hive.service.cli.thrift.ThriftCLIService.
> getSessionHandle(ThriftCLIService.java:413)
>         at org.apache.hive.service.cli.thrift.ThriftCLIService.
> OpenSession(ThriftCLIService.java:316)
>         at org.apache.hive.service.cli.thrift.TCLIService$Processor$
> OpenSession.getResult(TCLIService.java:1257)
>         at org.apache.hive.service.cli.thrift.TCLIService$Processor$
> OpenSession.getResult(TCLIService.java:1242)
>         at org.apache.thrift.ProcessFunction.process(
> ProcessFunction.java:39)
>         at org.apache.thrift.TBaseProcessor.process(
> TBaseProcessor.java:39)
>         at org.apache.thrift.server.TServlet.doPost(TServlet.java:83)
>         at org.apache.hive.service.cli.thrift.ThriftHttpServlet.
> doPost(ThriftHttpServlet.java:206)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>         at org.eclipse.jetty.servlet.ServletHolder.handle(
> ServletHolder.java:565)
>         at org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:479)
>         at org.eclipse.jetty.server.session.SessionHandler.
> doHandle(SessionHandler.java:225)
>         at org.eclipse.jetty.server.handler.ContextHandler.
> doHandle(ContextHandler.java:1031)
>         at org.eclipse.jetty.servlet.ServletHandler.doScope(
> ServletHandler.java:406)
>         at org.eclipse.jetty.server.session.SessionHandler.
> doScope(SessionHandler.java:186)
>         at org.eclipse.jetty.server.handler.ContextHandler.
> doScope(ContextHandler.java:965)
>         at org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:117)
>         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:111)
>         at org.eclipse.jetty.server.Server.handle(Server.java:349)
>         at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(
> AbstractHttpConnection.java:449)
>         at org.eclipse.jetty.server.AbstractHttpConnection$
> RequestHandler.content(AbstractHttpConnection.java:925)
>         at org.eclipse.jetty.http.HttpParser.parseNext(
> HttpParser.java:857)
>         at org.eclipse.jetty.http.HttpParser.parseAvailable(
> HttpParser.java:235)
>         at org.eclipse.jetty.server.AsyncHttpConnection.handle(
> AsyncHttpConnection.java:76)
>         at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(
> SelectChannelEndPoint.java:609)
>         at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(
> SelectChannelEndPoint.java:45)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.security.authorize.AuthorizationException:
> User: hue is not allowed to impersonate hdfs
>         at org.apache.hadoop.security.authorize.
> DefaultImpersonationProvider.authorize(DefaultImpersonationProvider.
> java:119)
>         at org.apache.hadoop.security.authorize.ProxyUsers.
> authorize(ProxyUsers.java:102)
>         at org.apache.hadoop.security.authorize.ProxyUsers.
> authorize(ProxyUsers.java:116)
>         at org.apache.hive.service.auth.HiveAuthFactory.verifyProxyAccess(
> HiveAuthFactory.java:392)
>         ... 32 more
> 2017-05-04 13:40:15,654 INFO  [HiveServer2-HttpHandler-Pool: Thread-60]:
> thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(313)) - Client
> protocol version: HIVE_CLI_SERVICE_PROTOCOL_V7
> 2017-05-04 13:40:15,658 WARN  [HiveServer2-HttpHandler-Pool: Thread-60]:
> thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(327)) - Error
> opening session:
> org.apache.hive.service.cli.HiveSQLException: Failed to validate proxy
> privilege of hue for hdfs
>         at org.apache.hive.service.auth.HiveAuthFactory.verifyProxyAccess(
> HiveAuthFactory.java:396)
>         at org.apache.hive.service.cli.thrift.ThriftCLIService.
> getProxyUser(ThriftCLIService.java:751)
>         at org.apache.hive.service.cli.thrift.ThriftCLIService.
> getUserName(ThriftCLIService.java:386)
>         at org.apache.hive.service.cli.thrift.ThriftCLIService.
> getSessionHandle(ThriftCLIService.java:413)
>         at org.apache.hive.service.cli.thrift.ThriftCLIService.
> OpenSession(ThriftCLIService.java:316)
>         at org.apache.hive.service.cli.thrift.TCLIService$Processor$
> OpenSession.getResult(TCLIService.java:1257)
>         at org.apache.hive.service.cli.thrift.TCLIService$Processor$
> OpenSession.getResult(TCLIService.java:1242)
>         at org.apache.thrift.ProcessFunction.process(
> ProcessFunction.java:39)
>         at org.apache.thrift.TBaseProcessor.process(
> TBaseProcessor.java:39)
>         at org.apache.thrift.server.TServlet.doPost(TServlet.java:83)
>         at org.apache.hive.service.cli.thrift.ThriftHttpServlet.
> doPost(ThriftHttpServlet.java:206)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>         at org.eclipse.jetty.servlet.ServletHolder.handle(
> ServletHolder.java:565)
>         at org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:479)
>         at org.eclipse.jetty.server.session.SessionHandler.
> doHandle(SessionHandler.java:225)
>         at org.eclipse.jetty.server.handler.ContextHandler.
> doHandle(ContextHandler.java:1031)
>         at org.eclipse.jetty.servlet.ServletHandler.doScope(
> ServletHandler.java:406)
>         at org.eclipse.jetty.server.session.SessionHandler.
> doScope(SessionHandler.java:186)
>         at org.eclipse.jetty.server.handler.ContextHandler.
> doScope(ContextHandler.java:965)
>         at org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:117)
>         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:111)
>         at org.eclipse.jetty.server.Server.handle(Server.java:349)
>         at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(
> AbstractHttpConnection.java:449)
>         at org.eclipse.jetty.server.AbstractHttpConnection$
> RequestHandler.content(AbstractHttpConnection.java:925)
>         at org.eclipse.jetty.http.HttpParser.parseNext(
> HttpParser.java:857)
>         at org.eclipse.jetty.http.HttpParser.parseAvailable(
> HttpParser.java:235)
>         at org.eclipse.jetty.server.AsyncHttpConnection.handle(
> AsyncHttpConnection.java:76)
>         at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(
> SelectChannelEndPoint.java:609)
>         at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(
> SelectChannelEndPoint.java:45)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.security.authorize.AuthorizationException:
> User: hue is not allowed to impersonate hdfs
>         at org.apache.hadoop.security.authorize.
> DefaultImpersonationProvider.authorize(DefaultImpersonationProvider.
> java:119)
>         at org.apache.hadoop.security.authorize.ProxyUsers.
> authorize(ProxyUsers.java:102)
>         at org.apache.hadoop.security.authorize.ProxyUsers.
> authorize(ProxyUsers.java:116)
>         at org.apache.hive.service.auth.HiveAuthFactory.verifyProxyAccess(
> HiveAuthFactory.java:392)
>         ... 32 more
>
>
>
> Please give me any ideas where to dig...
>
> Regards,
> Andrey
>
> 2017-04-20 23:04 GMT+03:00 Markovich <amrivkin@gmail.com>:
>
>> Hi Hive users,
>>
>> I've got a very strange problem and don't know where to go next, so
>> writting here, may be someone could help me.
>>
>> I've got HDP 2.5 with Hive 1.2.1000.2.5.0.0-1245 and Hadoop
>> 2.7.3.2.5.0.0-1245. I've got kerberos nad Ranger enabled.
>> I've installed HUE 3.11 on it, I'm getting erros like this: *Failed to
>> validate proxy privilege of hue for hdfs*, when logging into hue using
>> user hdfs.
>>
>> I've already added* hadoop.proxyuser.hue.groups=** and
>> *hadoop.proxyuser.hue.hosts=** in core-site.xml. Checked that this
>> settings were applied:
>>
>> # hadoop org.apache.hadoop.conf.Configuration | grep hue
>> <property><name>hadoop.proxyuser.hue.groups</name><value>*</
>> value><source>core-site.xml</source></property>
>> <property><name>hadoop.proxyuser.hue.hosts</name><value>*</
>> value><source>core-site.xml</source></property>
>>
>> Also checked properties like:*hive.server2.enable.impersonation *and
>> *hive.server2.enable.doAs.*
>> I've logged into beeline and connected to Hive using hue ticket:
>>
>> #klist
>> Ticket cache: FILE:/tmp/krb5cc_0
>> Default principal: hue@DEMO.TEST
>>
>> Valid starting       Expires              Service principal
>> 04/20/2017 19:40:50  04/21/2017 19:40:50  krbtgt/DEMO.TEST@DEMO.TEST
>>         renew until 04/27/2017 19:40:50
>>
>> #/usr/hdp/current/hive-client/bin/beeline --verbose
>> !connect jdbc:hive2://drm2.demo.test:10001/default;principal=hive/drm
>> 2.demo.test@DEMO.TEST;transportMode=http;httpPath=cliservice
>> ;hive.server2.proxy.user=hue
>>
>> 0: jdbc:hive2://drm2.demo.test:10001/defau> set
>> hive.server2.enable.impersonation;
>> Getting log thread is interrupted, since query is done!
>> +-----------------------------------------+--+
>> |                   set                   |
>> +-----------------------------------------+--+
>> | hive.server2.enable.impersonation=true  |
>> +-----------------------------------------+--+
>> 1 row selected (0.144 seconds)
>>
>> 0: jdbc:hive2://drm2.demo.test:10001/defau> set hive.server2.enable.doAs;
>> Getting log thread is interrupted, since query is done!
>> +--------------------------------+--+
>> |              set               |
>> +--------------------------------+--+
>> | hive.server2.enable.doAs=true  |
>> +--------------------------------+--+
>> 1 row selected (0.069 seconds)
>>
>> When I'm trying to use hdfs as proxyuser through beeline, I've get:
>> Connecting to jdbc:hive2://drm2.demo.test:10
>> 001/default;principal=hive/drm2.demo.test@DEMO.TEST;transpor
>> tMode=http;httpPath=cliservice;hive.server2.proxy.user=hdfs
>> Enter username for jdbc:hive2://drm2.demo.test:10
>> 001/default;principal=hive/drm2.demo.test@DEMO.TEST;transpor
>> tMode=http;httpPath=cliservice;hive.server2.proxy.user=hdfs:
>> Enter password for jdbc:hive2://drm2.demo.test:10
>> 001/default;principal=hive/drm2.demo.test@DEMO.TEST;transpor
>> tMode=http;httpPath=cliservice;hive.server2.proxy.user=hdfs:
>> Error: Failed to validate proxy privilege of hue for hdfs
>> (state=08S01,code=0)
>> org.apache.hive.service.cli.HiveSQLException: Failed to validate proxy
>> privilege of hue for hdfs
>> ...
>> Caused by: org.apache.hive.service.cli.HiveSQLException: Failed to
>> validate proxy privilege of hue for hdfs
>> ...
>> Caused by: org.apache.hadoop.security.authorize.AuthorizationException:
>> User: hue is not allowed to impersonate hdfs
>>
>> I've looked in Hadoop sources and this error means problem with
>> hadoop.proxyuser.hue.groups.
>> So at some very strange reasone hadoop is unable to allow user Hue to
>> impersonate hdfs or any other user.
>>
>> Where should I dig next? I'm a bit confused.
>>
>> Also yarn, hive, hdfs and hcat - all this users can impersonate any user,
>> so impersonation is working.
>> I've also checked if hadoop mapping to local is correct, and it seems to
>> be correct:
>> # hadoop org.apache.hadoop.security.HadoopKerberosName hue@DEMO.TEST
>> Name: hue@DEMO.TEST to hue
>>
>> Any ideas or help is welcome. I've stuck with this problem for 2 days
>> already.
>>
>> Regards,
>> Markovich
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>


-- 
王海华

Mime
View raw message