hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mohit Sabharwal (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-14187) JDOPersistenceManager objects remain cached if MetaStoreClient#close is not called
Date Thu, 07 Jul 2016 18:40:11 GMT

     [ https://issues.apache.org/jira/browse/HIVE-14187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mohit Sabharwal updated HIVE-14187:
-----------------------------------
    Description: 
JDOPersistenceManager objects are cached in JDOPersistenceManagerFactory by DataNuclues.

A new JDOPersistenceManager object gets created for every HMS thread since ObjectStore is
a thread local.

In non-embedded metastore mode, JDOPersistenceManager associated with a thread only gets cleaned
up if IMetaStoreClient#close is called by the client (which calls ObjectStore#shutdown which
calls JDOPersistenceManager#close which in turn removes the object from cache in JDOPersistenceManagerFactory#releasePersistenceManager
https://github.com/datanucleus/datanucleus-api-jdo/blob/master/src/main/java/org/datanucleus/api/jdo/JDOPersistenceManagerFactory.java#L1271),
i.e. the object will remain cached if client does not call close.

For example: If one interrupts out of hive CLI shell (instead of using 'exit;' command), SessionState#close
does not get called, and hence IMetaStoreClient#close does not get called.

Instead of relying the client to call close, it's cleaner to automatically perform RawStore
related cleanup at the server end via deleteContext() which gets called when the server detects
a lost/closed connection.

  was:
JDOPersistenceManager objects are cached in JDOPersistenceManagerFactory by DataNuclues.

A new JDOPersistenceManager object gets created for every HMS thread
local since ObjectStore is a thread local.

In non-embedded metastore mode, JDOPersistenceManager associated with a thread only gets cleaned
up if IMetaStoreClient#close is called by the client (which calls ObjectStore#shutdown which
calls JDOPersistenceManager#close which in turn removes the object from cache in JDOPersistenceManagerFactory#releasePersistenceManager
https://github.com/datanucleus/datanucleus-api-jdo/blob/master/src/main/java/org/datanucleus/api/jdo/JDOPersistenceManagerFactory.java#L1271),
i.e. the object will remain cached if client does not call close.

For example: If one interrupts out of hive CLI shell (instead of using 'exit;' command), SessionState#close
does not get called, and hence IMetaStoreClient#close does not get called.

Instead of relying the client to call close, it's cleaner to automatically perform RawStore
related cleanup at the server end via deleteContext() which gets called when the server detects
a lost/closed connection.


> JDOPersistenceManager objects remain cached if MetaStoreClient#close is not called
> ----------------------------------------------------------------------------------
>
>                 Key: HIVE-14187
>                 URL: https://issues.apache.org/jira/browse/HIVE-14187
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Mohit Sabharwal
>            Assignee: Mohit Sabharwal
>         Attachments: HIVE-14187.patch
>
>
> JDOPersistenceManager objects are cached in JDOPersistenceManagerFactory by DataNuclues.
> A new JDOPersistenceManager object gets created for every HMS thread since ObjectStore
is a thread local.
> In non-embedded metastore mode, JDOPersistenceManager associated with a thread only gets
cleaned up if IMetaStoreClient#close is called by the client (which calls ObjectStore#shutdown
which calls JDOPersistenceManager#close which in turn removes the object from cache in JDOPersistenceManagerFactory#releasePersistenceManager
> https://github.com/datanucleus/datanucleus-api-jdo/blob/master/src/main/java/org/datanucleus/api/jdo/JDOPersistenceManagerFactory.java#L1271),
i.e. the object will remain cached if client does not call close.
> For example: If one interrupts out of hive CLI shell (instead of using 'exit;' command),
SessionState#close does not get called, and hence IMetaStoreClient#close does not get called.
> Instead of relying the client to call close, it's cleaner to automatically perform RawStore
related cleanup at the server end via deleteContext() which gets called when the server detects
a lost/closed connection.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message