hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-10021) "Alter index rebuild" statements submitted through HiveServer2 fail when Sentry is enabled
Date Fri, 28 Aug 2015 23:39:46 GMT

    [ https://issues.apache.org/jira/browse/HIVE-10021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720785#comment-14720785
] 

Hive QA commented on HIVE-10021:
--------------------------------



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12753007/HIVE-10021.2.patch

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 9380 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropTable
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
{noformat}

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5105/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/5105/console
Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-5105/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12753007 - PreCommit-HIVE-TRUNK-Build

> "Alter index rebuild" statements submitted through HiveServer2 fail when Sentry is enabled
> ------------------------------------------------------------------------------------------
>
>                 Key: HIVE-10021
>                 URL: https://issues.apache.org/jira/browse/HIVE-10021
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2, Indexing
>    Affects Versions: 0.13.1, 2.0.0
>         Environment: CDH 5.3.2
>            Reporter: Richard Williams
>            Assignee: Aihua Xu
>         Attachments: HIVE-10021.2.patch, HIVE-10021.patch
>
>
> When HiveServer2 is configured to authorize submitted queries and statements through
Sentry, any attempt to issue an "alter index rebuild" statement fails with a SemanticException
caused by a NullPointerException. This occurs regardless of whether the index is a compact
or bitmap index. 
> The root cause of the problem appears to be the fact that the static createRootTask function
in org.apache.hadoop.hive.ql.optimizer.IndexUtils creates a new 
> org.apache.hadoop.hive.ql.Driver object to compile the index builder query, and this
new Driver object, unlike the one used by HiveServer2 to compile the submitted statement,
is used without having its "userName" field initialized with the submitting user's username.
Adding null checks to the Sentry code is insufficient to solve this problem, because Sentry
needs the userName to determine whether or not the submitting user should be able to execute
the index rebuild statement.
> Example stack trace from the HiveServer2 logs:
> {noformat}
> FAILED: NullPointerException null
> java.lang.NullPointerException
> 	at java.util.concurrent.ConcurrentHashMap.hash(ConcurrentHashMap.java:333)
> 	at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:988)
> 	at org.apache.hadoop.security.Groups.getGroups(Groups.java:161)
> 	at org.apache.sentry.provider.common.HadoopGroupMappingService.getGroups(HadoopGroupMappingService.java:46)
> 	at org.apache.sentry.binding.hive.authz.HiveAuthzBinding.getGroups(HiveAuthzBinding.java:370)
> 	at org.apache.sentry.binding.hive.HiveAuthzBindingHook.postAnalyze(HiveAuthzBindingHook.java:314)
> 	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:440)
> 	at org.apache.hadoop.hive.ql.optimizer.IndexUtils.createRootTask(IndexUtils.java:258)
> 	at org.apache.hadoop.hive.ql.index.compact.CompactIndexHandler.getIndexBuilderMapRedTask(CompactIndexHandler.java:149)
> 	at org.apache.hadoop.hive.ql.index.TableBasedIndexHandler.generateIndexBuildTaskList(TableBasedIndexHandler.java:67)
> 	at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.getIndexBuilderMapRed(DDLSemanticAnalyzer.java:1171)
> 	at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterIndexRebuild(DDLSemanticAnalyzer.java:1117)
> 	at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:410)
> 	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:204)
> 	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:437)
> 	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> 	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1026)
> 	at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1019)
> 	at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:100)
> 	at org.apache.hive.service.cli.operation.SQLOperation.run(SQLOperation.java:173)
> 	at org.apache.hive.service.cli.session.HiveSessionImpl.runOperationWithLogCapture(HiveSessionImpl.java:715)
> 	at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:370)
> 	at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:357)
> 	at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:238)
> 	at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:393)
> 	at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1373)
> 	at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1358)
> 	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> 	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> 	at org.apache.thrift.server.TServlet.doPost(TServlet.java:83)
> 	at org.apache.hive.service.cli.thrift.ThriftHttpServlet.doPost(ThriftHttpServlet.java:99)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> 	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:565)
> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:479)
> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:225)
> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1031)
> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:406)
> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)
> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:965)
> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
> 	at org.eclipse.jetty.server.Server.handle(Server.java:345)
> 	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:449)
> 	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:925)
> 	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857)
> 	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> 	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:76)
> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:609)
> 	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:45)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message