Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Thu, 27 Apr 2017 17:31:04 +0000 (UTC)
From: =?utf-8?Q?Sergio_Pe=C3=B1a_=28JIRA=29?= <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.13049818.1489105002000.68832.1493314264926@Atlassian.JIRA>
In-Reply-To: <JIRA.13049818.1489105002000@Atlassian.JIRA>
References: <JIRA.13049818.1489105002000@Atlassian.JIRA> <JIRA.13049818.1489105002696@jira-lw-us.apache.org>
Subject: [jira] [Updated] (HIVE-16164) Provide mechanism for passing HMS
 notification ID between transactional and non-transactional listeners.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Thu, 27 Apr 2017 17:31:12 -0000


     [ https://issues.apache.org/jira/browse/HIVE-16164?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sergio Pe=C3=B1a updated HIVE-16164:
-------------------------------
    Fix Version/s: 2.4.0

> Provide mechanism for passing HMS notification ID between transactional a=
nd non-transactional listeners.
> -------------------------------------------------------------------------=
-------------------------------
>
>                 Key: HIVE-16164
>                 URL: https://issues.apache.org/jira/browse/HIVE-16164
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore
>            Reporter: Sergio Pe=C3=B1a
>            Assignee: Sergio Pe=C3=B1a
>             Fix For: 2.3.0, 3.0.0, 2.4.0
>
>         Attachments: HIVE-16164.1.patch, HIVE-16164.2.patch, HIVE-16164.3=
.patch, HIVE-16164.6.patch, HIVE-16164.7.patch, HIVE-16164.8.patch
>
>
> The HMS DB notification listener currently stores an event ID on the HMS =
backend DB so that external applications (such as backup apps) can request =
incremental notifications based on the last event ID requested.
> The HMS DB notification and backup applications are asynchronous. However=
, there are sometimes that applications may be required to be in sync with =
the latest HMS event in order to process an action. These applications will=
 provide a listener implementation that is called by the HMS after an HMS t=
ransaction happened.
> The problem is that the listener running after the transaction (or during=
 the non-transactional context) may need the DB event ID in order to sync a=
ll events happened previous to that event ID, but this ID is never passed t=
o the non-transactional listeners.
> We can pass this event information through the EnvironmentContext found o=
n each ListenerEvent implementations (such as CreateTableEvent), and send t=
he EnvironmentContext to the non-transactional listeners to get the event I=
D.
> The DbNotificactionListener already knows the event ID after calling the =
ObjectStore.addNotificationEvent(). We just need to set this event ID to th=
e EnvironmentContext from each of the event notifications and make sure tha=
t this EnvironmentContext is sent to the non-transactional listeners.
> Here's the code example when creating a table on {{create_table_core}}:
> {noformat}
>  ms.createTable(tbl);
>   if (transactionalListeners.size() > 0) {
>     CreateTableEvent createTableEvent =3D new CreateTableEvent(tbl, true,=
 this);
>     createTableEvent.setEnvironmentContext(envContext);
>     for (MetaStoreEventListener transactionalListener : transactionalList=
eners) {
>       transactionalListener.onCreateTable(createTableEvent);         // <=
- Here the notification ID is generated
>     }
>   }
>   success =3D ms.commitTransaction();
> } finally {
>   if (!success) {
>     ms.rollbackTransaction();
>     if (madeDir) {
>       wh.deleteDir(tblPath, true);
>     }
>   }
>   for (MetaStoreEventListener listener : listeners) {
>     CreateTableEvent createTableEvent =3D
>         new CreateTableEvent(tbl, success, this);
>     createTableEvent.setEnvironmentContext(envContext);
>     listener.onCreateTable(createTableEvent);                        // <=
- Here we would like to consume notification ID
>   }
> {noformat}
> We could use a specific key name that will be used on the EnvironmentCont=
ext, such as DB_NOTIFICATION_EVENT_ID.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)