Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Thu, 6 Jul 2017 18:21:00 +0000 (UTC)
From: "Dan Burkert (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.13083959.1498868451000.187397.1499365260099@Atlassian.JIRA>
In-Reply-To: <JIRA.13083959.1498868451000@Atlassian.JIRA>
References: <JIRA.13083959.1498868451000@Atlassian.JIRA> <JIRA.13083959.1498868451542@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HIVE-17008) HiveMetastore.drop_database can
 return NPE if database does not exist
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Thu, 06 Jul 2017 18:21:05 -0000


    [ https://issues.apache.org/jira/browse/HIVE-17008?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D1607=
7005#comment-16077005 ]=20

Dan Burkert commented on HIVE-17008:
------------------------------------

My comment yesterday evening was a bit brief because I was low on time, but=
 here's the exact sequence of events that leads to the NPE which originally=
 prompted this issue.  As we've discussed it's just one of many inter-relat=
ed issues in the class, but I wanted to make it clear what's happening in t=
his specific case:

1. Application calls {{ThriftHiveMetastore.drop_database}} with a non-exist=
ent database name via the HMS thrift API=C2=B9.
2. In {{HiveMetaStore.drop_database_core}}, the {{db}} local variable is [i=
nitialized to {{null}}|https://github.com/apache/hive/blob/555f001146c4fc47=
1e29e18899a0e02a4043cca5/metastore/src/java/org/apache/hadoop/hive/metastor=
e/HiveMetaStore.java#L1023].
3. In {{HiveMetaStore.drop_database_core}}, the [lookup of the non-existent=
 database fails|https://github.com/apache/hive/blob/555f001146c4fc471e29e18=
899a0e02a4043cca5/metastore/src/java/org/apache/hadoop/hive/metastore/HiveM=
etaStore.java#L1029], leaving {{db}} set to {{null}} and unwinding to the {=
{finally}} block.
4. In {{HiveMetaSotre.drop_database_core}} {{finally}} block, a new [{{Drop=
DatabaseEvent}} is created|https://github.com/apache/hive/blob/555f001146c4=
fc471e29e18899a0e02a4043cca5/metastore/src/java/org/apache/hadoop/hive/meta=
store/HiveMetaStore.java#L1139-L1143] with the {{null}} database, and liste=
ners are notified with this event.
5. Somewhere (I haven't traced this bit), the notification log event listen=
er is dereferencing the null database.

=C2=B9 Although I haven't reproduced it, it should be possible to reproduce=
 this bug through the Java {{HiveMetastoreClient}} API as well, but it woul=
d require concurrent DDL operations.  The {{HiveMetastoreClient}} [checks t=
hat the database exists|https://github.com/apache/hive/blob/master/metastor=
e/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java#L851-L=
858] before attempting to drop it; this is a benign TOCTOU which makes it d=
ifficult to reproduce using that API (again, it should still possible with =
the right interleavings of concurrent DDL ops).  A light skimming through t=
hat class reveals that the client is _very_ aggressive about validating sta=
te exists before issuing DDL operations.  Just my opinion, but that's a big=
 code smell; the client should rely on the server to validate arguments.

> HiveMetastore.drop_database can return NPE if database does not exist
> ---------------------------------------------------------------------
>
>                 Key: HIVE-17008
>                 URL: https://issues.apache.org/jira/browse/HIVE-17008
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>            Reporter: Dan Burkert
>            Assignee: Dan Burkert
>         Attachments: HIVE-17008.0.patch
>
>
> When dropping a non-existent database, the HMS will still fire registered=
 {{DROP_DATABASE}} event listeners.  This results in an NPE when the listen=
ers attempt to deref the {{null}} database parameter.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)