hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-18778) Needs to capture input/output entities in explain
Date Sat, 29 Sep 2018 14:54:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-18778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16633037#comment-16633037
] 

Hive QA commented on HIVE-18778:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12941769/HIVE-18778.12.branch-3.1.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 103 failed/errored test(s), 14379 tests executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=271)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestCatalogs - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestDatabaseName - did not produce a TEST-*.xml file (likely timed out) (batchId=195)
TestDatabases - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestDeadline - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=271)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed out) (batchId=229)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestForeignKey - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestFunctions - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestGetPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file (likely timed
out) (batchId=231)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestHyperLogLogSparse - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestJSONMessageDeserializer - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestListPartitions - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestLockRequestBuilder - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestMarkPartition - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestMarkPartitionRemote - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestMetaStoreConnectionUrlHook - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestMetaStoreEndFunctionListener - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestMetaStoreEventListener - did not produce a TEST-*.xml file (likely timed out) (batchId=232)
TestMetaStoreEventListenerOnlyOnCommit - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestMetaStoreEventListenerWithOldConf - did not produce a TEST-*.xml file (likely timed out)
(batchId=236)
TestMetaStoreInitListener - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestMetaStoreListenersError - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestMetaStoreSchemaFactory - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestMetaStoreSchemaInfo - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestMetaStoreServerUtils - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestMetastoreConf - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestMetastoreSchemaTool - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestMetrics - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=271)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=271)
TestNotNullConstraint - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestObjectStore - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestObjectStoreInitRetry - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestObjectStoreSchemaMethods - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestOldSchema - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestPartitionNameWhitelistValidation - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestPreUpgradeTool - did not produce a TEST-*.xml file (likely timed out) (batchId=329)
TestPrimaryKey - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestRawStoreProxy - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestRemoteHiveMetaStore - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
TestRemoteHiveMetaStoreIpAddress - did not produce a TEST-*.xml file (likely timed out) (batchId=224)
TestRemoteUGIHiveMetaStoreIpAddress - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestRetriesInRetryingHMSHandler - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestRetryingHMSHandler - did not produce a TEST-*.xml file (likely timed out) (batchId=234)
TestRuntimeStats - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestSchemaToolForMetastore - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestSetUGIOnBothClientServer - did not produce a TEST-*.xml file (likely timed out) (batchId=227)
TestSetUGIOnOnlyClient - did not produce a TEST-*.xml file (likely timed out) (batchId=225)
TestSetUGIOnOnlyServer - did not produce a TEST-*.xml file (likely timed out) (batchId=235)
TestSparseEncodeHash - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestStats - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestStatsSetupConst - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestTableIterable - did not produce a TEST-*.xml file (likely timed out) (batchId=236)
TestTableName - did not produce a TEST-*.xml file (likely timed out) (batchId=195)
TestTablesCreateDropAlterTruncate - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestTablesGetExists - did not produce a TEST-*.xml file (likely timed out) (batchId=228)
TestTablesList - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=271)
TestTxnHandlerNegative - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestTxnUtils - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
TestUniqueConstraint - did not produce a TEST-*.xml file (likely timed out) (batchId=226)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan] (batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_with_masking]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[strict_managed_tables_sysdb]
(batchId=169)
org.apache.hadoop.hive.metastore.client.TestRuntimeStats.testCleanup[Remote] (batchId=213)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testManagedPaths (batchId=248)
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation (batchId=257)
org.apache.hive.service.server.TestInformationSchemaWithPrivilege.test (batchId=255)
org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=323)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14126/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14126/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14126/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 103 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12941769 - PreCommit-HIVE-Build

> Needs to capture input/output entities in explain
> -------------------------------------------------
>
>                 Key: HIVE-18778
>                 URL: https://issues.apache.org/jira/browse/HIVE-18778
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>            Priority: Major
>         Attachments: HIVE-18778-SparkPositive.patch, HIVE-18778.1.patch, HIVE-18778.10.branch-3.patch,
HIVE-18778.11.branch-3.1.patch, HIVE-18778.11.branch-3.patch, HIVE-18778.12.branch-3.1.patch,
HIVE-18778.2.patch, HIVE-18778.3.patch, HIVE-18778.4.patch, HIVE-18778.5.patch, HIVE-18778.6.patch,
HIVE-18778.7.patch, HIVE-18778.8.patch, HIVE-18778.9.branch-3.patch, HIVE-18778.9.patch, HIVE-18778_TestCliDriver.patch,
HIVE-18788_SparkNegative.patch, HIVE-18788_SparkPerf.patch
>
>
> With Sentry enabled, commands like explain drop table foo fail with {{explain drop table
foo;}}
> {code}
> Error: Error while compiling statement: FAILED: SemanticException No valid privileges
>  Required privilege( Table) not available in input privileges
>  The required privileges: (state=42000,code=40000)
> {code}
> Sentry fails to authorize because the ExplainSemanticAnalyzer uses an instance of DDLSemanticAnalyzer
to analyze the explain query.
> {code}
> BaseSemanticAnalyzer sem = SemanticAnalyzerFactory.get(conf, input);
> sem.analyze(input, ctx);
> sem.validate()
> {code}
> The inputs/outputs entities for this query are set in the above code. However, these
are never set on the instance of ExplainSemanticAnalyzer itself and thus is not propagated
into the HookContext in the calling Driver code.
> {code}
> sem.analyze(tree, ctx); --> this results in calling the above code that uses DDLSA
> hookCtx.update(sem); --> sem is an instance of ExplainSemanticAnalyzer, this code
attempts to update the HookContext with the input/output info from ESA which is never set.
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message