hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sushanth Sowmyan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-12937) DbNotificationListener unable to clean up old notification events
Date Thu, 24 Mar 2016 02:28:25 GMT

     [ https://issues.apache.org/jira/browse/HIVE-12937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sushanth Sowmyan updated HIVE-12937:
------------------------------------
    Attachment: HIVE-12937.patch

Patch attached.

> DbNotificationListener unable to clean up old notification events
> -----------------------------------------------------------------
>
>                 Key: HIVE-12937
>                 URL: https://issues.apache.org/jira/browse/HIVE-12937
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 1.3.0, 1.2.1, 2.0.0, 2.1.0
>            Reporter: Sushanth Sowmyan
>            Assignee: Sushanth Sowmyan
>         Attachments: HIVE-12937.patch
>
>
> There is a bug in ObjectStore, where we use pm.deletePersistent instead of pm.deletePersistentAll,
which causes the persistenceManager to try and drop a org.datanucleus.store.rdbms.query.ForwardQueryResult
instead of the appropriate associated org.apache.hadoop.hive.metastore.model.MNotificationLog.
> This results in an error that looks like this:
> {noformat}
> Exception in thread "CleanerThread" org.datanucleus.api.jdo.exceptions.ClassNotPersistenceCapableException:
The class "org.datanucleus.store.rdbms.query.ForwardQueryResult" is not persistable. This
means that it either hasnt been enhanced, or that the enhanced version of the file is not
in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for
the class are not found.
> at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:380)
> at org.datanucleus.api.jdo.JDOPersistenceManager.jdoDeletePersistent(JDOPersistenceManager.java:807)
> at org.datanucleus.api.jdo.JDOPersistenceManager.deletePersistent(JDOPersistenceManager.java:820)
> at org.apache.hadoop.hive.metastore.ObjectStore.cleanNotificationEvents(ObjectStore.java:7149)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
> at com.sun.proxy.$Proxy0.cleanNotificationEvents(Unknown Source)
> at org.apache.hive.hcatalog.listener.DbNotificationListener$CleanerThread.run(DbNotificationListener.java:277)
> NestedThrowablesStackTrace:
> The class "org.datanucleus.store.rdbms.query.ForwardQueryResult" is not persistable.
This means that it either hasnt been enhanced, or that the enhanced version of the file is
not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations
for the class are not found.
> org.datanucleus.exceptions.ClassNotPersistableException: The class "org.datanucleus.store.rdbms.query.ForwardQueryResult"
is not persistable. This means that it either hasnt been enhanced, or that the enhanced version
of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations
for the class are not found.
> at org.datanucleus.ExecutionContextImpl.assertClassPersistable(ExecutionContextImpl.java:5698)
> at org.datanucleus.ExecutionContextImpl.deleteObjectInternal(ExecutionContextImpl.java:2495)
> at org.datanucleus.ExecutionContextImpl.deleteObjectWork(ExecutionContextImpl.java:2466)
> at org.datanucleus.ExecutionContextImpl.deleteObject(ExecutionContextImpl.java:2417)
> at org.datanucleus.ExecutionContextThreadedImpl.deleteObject(ExecutionContextThreadedImpl.java:245)
> at org.datanucleus.api.jdo.JDOPersistenceManager.jdoDeletePersistent(JDOPersistenceManager.java:802)
> at org.datanucleus.api.jdo.JDOPersistenceManager.deletePersistent(JDOPersistenceManager.java:820)
> at org.apache.hadoop.hive.metastore.ObjectStore.cleanNotificationEvents(ObjectStore.java:7149)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:114)
> at com.sun.proxy.$Proxy0.cleanNotificationEvents(Unknown Source)
> at org.apache.hive.hcatalog.listener.DbNotificationListener$CleanerThread.run(DbNotificationListener.java:277)
> {noformat}
> The end result of this bug is that users of DbNotificationListener will have an evergrowing
number of notification events that are not cleaned up as they age. This is an easy enough
fix, but shows that we have a lack of test coverage here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message