hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16979) Cache UGI for metastore
Date Thu, 29 Jun 2017 19:49:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16068871#comment-16068871
] 

Hive QA commented on HIVE-16979:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12874981/HIVE-16979.2.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10831 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main] (batchId=150)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=233)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testBootstrapFunctionReplication
(batchId=217)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testCreateFunctionIncrementalReplication
(batchId=217)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testCreateFunctionWithFunctionBinaryJarsOnHDFS
(batchId=217)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=178)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=178)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=178)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5832/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5832/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5832/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12874981 - PreCommit-HIVE-Build

> Cache UGI for metastore
> -----------------------
>
>                 Key: HIVE-16979
>                 URL: https://issues.apache.org/jira/browse/HIVE-16979
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Tao Li
>            Assignee: Tao Li
>         Attachments: HIVE-16979.1.patch, HIVE-16979.2.patch
>
>
> FileSystem.closeAllForUGI is called per request against metastore to dispose UGI, which
involves talking to HDFS name node and is time consuming. So the perf improvement would be
caching and reusing the UGI.
> Per FileSystem.closeAllForUG call could take up to 20 ms as E2E latency against HDFS.
Usually a Hive query could result in several calls against metastore, so we can save up to
50-100 ms per hive query.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message