drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4768) Drill may leak hive meta store connection if hive meta store client call hits error
Date Tue, 12 Jul 2016 00:42:11 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371984#comment-15371984
] 

ASF GitHub Bot commented on DRILL-4768:
---------------------------------------

Github user jinfengni commented on a diff in the pull request:

    https://github.com/apache/drill/pull/543#discussion_r70361935
  
    --- Diff: contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/DrillHiveMetaStoreClient.java
---
    @@ -209,8 +213,10 @@ private DrillHiveMetaStoreClient(final HiveConf hiveConf) throws
MetaException {
           throws TException {
         try {
           return mClient.getAllTables(dbName);
    +    } catch (MetaException | UnknownDBException e) {
    +      throw e;
         } catch (TException e) {
    -      logger.warn("Failure while attempting to get hive tables", e);
    +      logger.warn("Failure while attempting to get hive tables. Retries once.", e);
    --- End diff --
    
    Add code to check if close() hits exception. In case close() fails with exception, we
log the exception and continue to reconnect(). This may leak connection. 
    
    However, we have to maintain hive metastore connection. Otherwise, next hive query may
fail, and the only way to continue is either re-enable hive storage plugin (which may then
leak a connection DRILL-4774), or re-start drillbit.



> Drill may leak hive meta store connection if hive meta store client call hits error
> -----------------------------------------------------------------------------------
>
>                 Key: DRILL-4768
>                 URL: https://issues.apache.org/jira/browse/DRILL-4768
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - Hive
>            Reporter: Jinfeng Ni
>            Assignee: Jinfeng Ni
>
> We are seeing one drillbit creates hundreds of connections to hive meta store. This indicates
that drill is leaking those connection, and did not close those connections properly. When
such leaking happens, it may prevent other applications from connecting to hive meta store.

> It seems one cause of leaking connection happens when hive meta store client call hits
exception. 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message