hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20953) Remove a function from function registry when it can not be added to the metastore when creating it.
Date Fri, 30 Nov 2018 07:51:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704372#comment-16704372
] 

Hive QA commented on HIVE-20953:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12950123/HIVE-20953.03

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15641 tests executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=196)
	[druidmini_dynamic_partition.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_test_insert.q,druidkafkamini_delimited.q]
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join4]
(batchId=192)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/15126/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15126/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15126/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12950123 - PreCommit-HIVE-Build

> Remove a function from function registry when it can not be added to the metastore when
creating it.
> ----------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-20953
>                 URL: https://issues.apache.org/jira/browse/HIVE-20953
>             Project: Hive
>          Issue Type: Bug
>          Components: Tests
>    Affects Versions: 4.0.0
>            Reporter: Ashutosh Bapat
>            Assignee: Ashutosh Bapat
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>         Attachments: HIVE-20953.01, HIVE-20953.02, HIVE-20953.02, HIVE-20953.03, test_func_load_failure_retry.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> The testcase is intended to test REPL LOAD with retry. The test creates a partitioned
table and a function in the source database and loads those to the replica. The first attempt
to load a dump is intended to fail while loading one of the partitions. Based on the order
in which the objects get loaded, if the function is queued after the table, it will not be
available in replica after the load failure. But if it's queued before the table, it will
be available in replica even after the load failure. The test assumes the later case, which
may not be true always.
> Hence fix the testcase to order the objects by a fixed ordering. By setting hive.in.repl.test.files.sorted
to true, the objects are ordered by the directory names. This ordering is available with
minimal changes for testing, hence we use it. With this ordering a function gets loaded before
a table. So changed the test to not expect the function to be available after the failed
load, but be available after the retry.
> While writing that testcase, I found that even if a function fails to load, it's visible
through show functions and also is available to be called just as if the failure has not happened.
Digging further it was found that when creating a function we add it to the registry and also
to the metastore. If the later fails, we do not clean it up from the registry and thus it
remains visible after failure. Fixed the same.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message