hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "anishek (JIRA)" <>
Subject [jira] [Commented] (HIVE-16738) Notification ID generation in DBNotification might not be unique across HS2 instances.
Date Wed, 24 May 2017 05:23:04 GMT


anishek commented on HIVE-16738:

[~mohitsabharwal], i am looking at use cases where we have two separate instances of HS2 running
in different machines(say HS1 and HS2)  with both of them connecting to a common hive metastore
and the clients re connecting to one of the hive server 2 machines randomly, 

client 1 connects to HS1
client 2 connects to hs2

threads on HS1 and HS2 should be able to get a lock on {{NOTIFICATION_TBL_LOCK }} at the same
time => hence entering the critical section to update the notification Event which would
read the notification sequence entry with primary key 1 in both cases and update to the same
value for possibly two events ?

> Notification ID generation in DBNotification might not be unique across HS2 instances.
> --------------------------------------------------------------------------------------
>                 Key: HIVE-16738
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 3.0.0
>            Reporter: anishek
>            Assignee: anishek
>             Fix For: 3.0.0
> Going to explain the problem in scope of "replication" feature for hive 2 that is being
built, as it is easier to explain:
> To allow replication to work we need to set "hive.metastore.transactional.event.listeners"
 to DBNotificationListener. For use cases where there are multiple HiveServer2 Instances running

> {code}
>  private void process(NotificationEvent event, ListenerEvent listenerEvent) throws MetaException
>     event.setMessageFormat(msgFactory.getMessageFormat());
>     synchronized (NOTIFICATION_TBL_LOCK) {
>       LOG.debug("DbNotificationListener: Processing : {}:{}", event.getEventId(),
>           event.getMessage());
>       HMSHandler.getMSForConf(hiveConf).addNotificationEvent(event);
>     }
>       // Set the DB_NOTIFICATION_EVENT_ID for future reference by other listeners.
>       if (event.isSetEventId()) {
>         listenerEvent.putParameter(
>             MetaStoreEventListenerConstants.DB_NOTIFICATION_EVENT_ID_KEY_NAME,
>             Long.toString(event.getEventId()));
>       }
>   }
> {code}
> the above code in DBNotificationListner having the object lock wont be guarantee enough
to make sure that all events get a unique id. The transaction isolation level at the db "read-comitted"
or "repeatable-read"  would  also not guarantee the same, unless a lock is at the db level
preferably on table {{NOTIFICATION_SEQUENCE}} which only has one row.

This message was sent by Atlassian JIRA

View raw message