hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dan Burkert (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17008) HiveMetastore.drop_database can return NPE if database does not exist
Date Thu, 06 Jul 2017 18:21:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16077005#comment-16077005
] 

Dan Burkert commented on HIVE-17008:
------------------------------------

My comment yesterday evening was a bit brief because I was low on time, but here's the exact
sequence of events that leads to the NPE which originally prompted this issue.  As we've discussed
it's just one of many inter-related issues in the class, but I wanted to make it clear what's
happening in this specific case:

1. Application calls {{ThriftHiveMetastore.drop_database}} with a non-existent database name
via the HMS thrift API¹.
2. In {{HiveMetaStore.drop_database_core}}, the {{db}} local variable is [initialized to {{null}}|https://github.com/apache/hive/blob/555f001146c4fc471e29e18899a0e02a4043cca5/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L1023].
3. In {{HiveMetaStore.drop_database_core}}, the [lookup of the non-existent database fails|https://github.com/apache/hive/blob/555f001146c4fc471e29e18899a0e02a4043cca5/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L1029],
leaving {{db}} set to {{null}} and unwinding to the {{finally}} block.
4. In {{HiveMetaSotre.drop_database_core}} {{finally}} block, a new [{{DropDatabaseEvent}}
is created|https://github.com/apache/hive/blob/555f001146c4fc471e29e18899a0e02a4043cca5/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L1139-L1143]
with the {{null}} database, and listeners are notified with this event.
5. Somewhere (I haven't traced this bit), the notification log event listener is dereferencing
the null database.

¹ Although I haven't reproduced it, it should be possible to reproduce this bug through the
Java {{HiveMetastoreClient}} API as well, but it would require concurrent DDL operations.
 The {{HiveMetastoreClient}} [checks that the database exists|https://github.com/apache/hive/blob/master/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java#L851-L858]
before attempting to drop it; this is a benign TOCTOU which makes it difficult to reproduce
using that API (again, it should still possible with the right interleavings of concurrent
DDL ops).  A light skimming through that class reveals that the client is _very_ aggressive
about validating state exists before issuing DDL operations.  Just my opinion, but that's
a big code smell; the client should rely on the server to validate arguments.

> HiveMetastore.drop_database can return NPE if database does not exist
> ---------------------------------------------------------------------
>
>                 Key: HIVE-17008
>                 URL: https://issues.apache.org/jira/browse/HIVE-17008
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>            Reporter: Dan Burkert
>            Assignee: Dan Burkert
>         Attachments: HIVE-17008.0.patch
>
>
> When dropping a non-existent database, the HMS will still fire registered {{DROP_DATABASE}}
event listeners.  This results in an NPE when the listeners attempt to deref the {{null}}
database parameter.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message