hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vaibhav Gumashta (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-8761) JDOPersistenceManager creation should be controlled by at the Server level and not Thread level
Date Thu, 06 Nov 2014 20:25:34 GMT
Vaibhav Gumashta created HIVE-8761:
--------------------------------------

             Summary: JDOPersistenceManager creation should be controlled by at the Server
level and not Thread level
                 Key: HIVE-8761
                 URL: https://issues.apache.org/jira/browse/HIVE-8761
             Project: Hive
          Issue Type: Bug
          Components: Metastore
    Affects Versions: 0.15.0
            Reporter: Vaibhav Gumashta
            Assignee: Vaibhav Gumashta


When using JDO, we create a thread local RawStore (ObjectStore) object in each metastore thread.
This leads to creation of a new  JDOPersistenceManager per thread which are cached in JDOPersistanceManagerFactory.
To remove JDOPersistenceManager from JDOPersistanceManagerFactory, an explicit JDOPersistenceManager.close
needs to be called. 
This is a bad candidate for thread local as the effective object destruction requires the
application to call close. So, when metastore threads are killed by the threadpool, this object
will never be removed from the JDOPersistanceManagerFactory cache.
We fixed this for HiveServer2 using embedded metastore (HIVE-7353) by customizing the GC collection
of the dying thread, but I believe a better and more efficient solution is to pool JDOPersistenceManager
objects and let each thread get an object for its use from the pool.
 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message