hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17169) Avoid extra call to KeyProvider::getMetadata()
Date Fri, 28 Jul 2017 13:58:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16105001#comment-16105001
] 

Hive QA commented on HIVE-17169:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12878839/HIVE-17169.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 11013 tests executed
*Failed tests:*
{noformat}
TestPerfCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=235)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=240)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=144)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=168)
org.apache.hadoop.hive.metastore.TestHiveMetaStoreStatsMerge.testStatsMerge (batchId=206)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema (batchId=179)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=179)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=179)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6169/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6169/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6169/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12878839 - PreCommit-HIVE-Build

> Avoid extra call to KeyProvider::getMetadata()
> ----------------------------------------------
>
>                 Key: HIVE-17169
>                 URL: https://issues.apache.org/jira/browse/HIVE-17169
>             Project: Hive
>          Issue Type: Bug
>          Components: Shims
>    Affects Versions: 3.0.0
>            Reporter: Mithun Radhakrishnan
>            Assignee: Mithun Radhakrishnan
>         Attachments: HIVE-17169.1.patch
>
>
> Here's the code from {{Hadoop23Shims}}:
> {code:title=Hadoop23Shims.java|borderStyle=solid}
>     @Override
>     public int comparePathKeyStrength(Path path1, Path path2) throws IOException {
>       EncryptionZone zone1, zone2;
>       zone1 = hdfsAdmin.getEncryptionZoneForPath(path1);
>       zone2 = hdfsAdmin.getEncryptionZoneForPath(path2);
>       if (zone1 == null && zone2 == null) {
>         return 0;
>       } else if (zone1 == null) {
>         return -1;
>       } else if (zone2 == null) {
>         return 1;
>       }
>       return compareKeyStrength(zone1.getKeyName(), zone2.getKeyName());
>     }
>     private int compareKeyStrength(String keyname1, String keyname2) throws IOException
{
>       KeyProvider.Metadata meta1, meta2;
>       if (keyProvider == null) {
>         throw new IOException("HDFS security key provider is not configured on your server.");
>       }
>       meta1 = keyProvider.getMetadata(keyname1);
>       meta2 = keyProvider.getMetadata(keyname2);
>       if (meta1.getBitLength() < meta2.getBitLength()) {
>         return -1;
>       } else if (meta1.getBitLength() == meta2.getBitLength()) {
>         return 0;
>       } else {
>         return 1;
>       }
>     }
>   }
> {code}
> It turns out that {{EncryptionZone}} already has the cipher's bit-length stored in a
member variable. One shouldn't need an additional name-node call ({{KeyProvider::getMetadata()}})
only to fetch it again.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message