hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunitha Beeram (JIRA)" <>
Subject [jira] [Commented] (HIVE-16908) Failures in TestHcatClient due to HIVE-16844
Date Wed, 21 Jun 2017 22:08:01 GMT


Sunitha Beeram commented on HIVE-16908:

[~sankarh] would be great if you reviewed the patch or recommended someone else more familiar
with the code/original changes to also review the changes. 
Hopefully I haven't oversimplified or relied on other/incorrect assumptions that make the
tests pass currently.

I am including the description from the RB here for quick reference:
"HIVE-16844 fixed a connection leak issue which subsequently exposed failures in TestHCatClient.
The connection leak gets triggered if a metastore instance is updated with a different JDO
configuration. TestHCatClient uses 2 metastore instances to test replication related methods.
Unfortunately, it does so by providing a different derby db name for the second instance.
Since the 2 metastores run in the same JVM, the path fixed in HIVE-16844 gets triggered, resulting
in "sourceMetastore"'s connection being closed and thus resulting in failures.

It appears to me that running 2 metastore instances within the same JVM is error prone as
there could be unintentional side-effects due to statics in the code (as was exposed by fixing
HIVE-16844). This patch provides a way to test the replication related methods without involving
a second instance. The changes mainly validate the serialize/deserialize methods. One of the
tests, testPartitionRegistrationWithCustomSchema, uses addPartitions method to verify propogation
of changes and it appeared that addPartitions wasn't covered by other tests in TestHCatClient
and there wasn't a better way to verify the intended path, so I used an approach where the
original database and table are dropped and recreated using the serialized-string and captured
partition spec."

> Failures in TestHcatClient due to HIVE-16844
> --------------------------------------------
>                 Key: HIVE-16908
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sunitha Beeram
>            Assignee: Sunitha Beeram
>         Attachments: HIVE-16908.1.patch
> Some of the tests in, for ex:
> {noformat}
> org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
> org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
> org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation (batchId=177)
> {noformat}
> are failing due to HIVE-16844. HIVE-16844 fixes a connection leak when a new configuration
object is set on the ObjectStore. TestHCatClient fires up a second instance of metastore thread
with a different conf object that results in the PersistenceMangaerFactory closure and hence
tests fail. 

This message was sent by Atlassian JIRA

View raw message