phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yeonseop Kim (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3553) Zookeeper connection should be closed immediately after DefaultStatisticsCollector's collecting stats done
Date Tue, 03 Jan 2017 04:44:58 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15794129#comment-15794129
] 

Yeonseop Kim commented on PHOENIX-3553:
---------------------------------------

I think there is no need of test code, because the patch is trivial. I only added try-finally
wrapping and htable.close().
RegionCoproceesorEnvironment.getTable calls CoprocessorHConnection.getConnectionForEnvironment
which returns ClusterConnection object. 
ClusterConnection is an unmanaged connection which we must clean up externally, as mentioned
in hbase source ( see line 54 in https://github.com/apache/hbase/blob/branch-1.1/hbase-server/src/main/java/org/apache/hadoop/hbase/client/CoprocessorHConnection.java)

> Zookeeper connection should be closed immediately after DefaultStatisticsCollector's
collecting stats done
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3553
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3553
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.9.0
>            Reporter: Yeonseop Kim
>              Labels: stats, zookeeper
>             Fix For: 4.10.0
>
>         Attachments: PHOENIX-3553.patch
>
>
> In every minor compaction job of HBase,
> org.apache.phoenix.schema.stats.DefaultStatisticsCollector.initGuidePostDepth() is called,
> and SYSTEM.CATALOG table is open to get guidepost width via
> htable = env.getTable(
>                  SchemaUtil.getPhysicalTableName(PhoenixDatabaseMetaData.SYSTEM_CATALOG_NAME_BYTES,
env.getConfiguration()));
> This function call creates one zookeeper connection to get cluster id.
> DefaultStatisticsCollector doesn't close this zookeeper connection immediately after
get guidepost width, and the zookeeper connection remains alive until HRegion is closed.
> This is not a problem with small number of Regions, but when number of Region is large
and upsert operation is frequent, the number of zookeeper connection gradually increases 
to hundreds, and the zookeeper server nodes experience  short of available TCP/IP ports.
> This zookeeper connection should be closed immediately after get guidepost width.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message