phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From István Tóth (JIRA) <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-5311) Integration tests leak tables when running on distributed cluster
Date Thu, 06 Jun 2019 10:44:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16857515#comment-16857515
] 

István Tóth edited comment on PHOENIX-5311 at 6/6/19 10:43 AM:
---------------------------------------------------------------

Yes, I am doing exactly that.

I have attached a patch. 

There is no need for additional configuration, as there is already a method that can tell
if the tests are running on a real cluster. 

Instead of dropping the hbase tables, I have chosen to drop the phoenix tables, and re-enable 

phoenix.schema.dropMetaData in the distributed case. 

With these changes I am able to successfully run End2EndTestDriver on a modestly sized test
cluster, without running out of resources.


was (Author: stoty):
Yes, I am doing exactly that.

I have attached a patch. 

There is no need for additional configuration, as there is already a method that can tell
if the tests are running on a real cluster. 

Instead of dropping the hbase tables, I have chosen to drop the phoenix tables, and re-enable 

phoenix.schema.dropMetaData in the distributed case. 

With these changes I am able to successfully run End2EndTestDriver on a modestly sized real
test cluster, without running out of resources.

> Integration tests leak tables when running on distributed cluster
> -----------------------------------------------------------------
>
>                 Key: PHOENIX-5311
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5311
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: István Tóth
>            Priority: Major
>         Attachments: PHOENIX-5311.master.v1.patch
>
>
> When integration test suite is run via End2EndTestDriver on a distributed cluster, most
tests do not clean up their tables, leaving thousands of tables on the cluster, and exhausting
RegionServer memory.
> There are actually three problems:
>  * The BaseTest.freeResourcesIfBeyondThreshold() method is called after most tests, and
it restarts the MiniCluster, thus freeing resources, but it has no effect when running on
a distributed cluster.
>  * The TestDriver sets phoenix.schema.dropMetaData to false by default, so even if the
Phoenix tables are dropped, the HBASE tables are not, so the table leak remains. 
>  * The phoenix.schema.dropMetaData setting cannot be easily overridden because of PHOENIX-5310
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message