hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "anishek (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-16738) Notification ID generation in DBNotification might not be unique across HS2 instances.
Date Tue, 23 May 2017 06:54:04 GMT

     [ https://issues.apache.org/jira/browse/HIVE-16738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

anishek updated HIVE-16738:
---------------------------
    Summary: Notification ID generation in DBNotification might not be unique across HS2 instances.
 (was: Notification ID generation in DBNotification might not be unique)

> Notification ID generation in DBNotification might not be unique across HS2 instances.
> --------------------------------------------------------------------------------------
>
>                 Key: HIVE-16738
>                 URL: https://issues.apache.org/jira/browse/HIVE-16738
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 3.0.0
>            Reporter: anishek
>            Assignee: anishek
>             Fix For: 3.0.0
>
>
> Going to explain the problem in scope of "replication" feature for hive 2 that is being
built, as it is easier to explain:
> To allow replication to work we need to set "hive.metastore.transactional.event.listeners"
 to DBNotificationListener. For use cases where there are multiple HiveServer2 Instances running

> {code}
>  private void process(NotificationEvent event, ListenerEvent listenerEvent) throws MetaException
{
>     event.setMessageFormat(msgFactory.getMessageFormat());
>     synchronized (NOTIFICATION_TBL_LOCK) {
>       LOG.debug("DbNotificationListener: Processing : {}:{}", event.getEventId(),
>           event.getMessage());
>       HMSHandler.getMSForConf(hiveConf).addNotificationEvent(event);
>     }
>       // Set the DB_NOTIFICATION_EVENT_ID for future reference by other listeners.
>       if (event.isSetEventId()) {
>         listenerEvent.putParameter(
>             MetaStoreEventListenerConstants.DB_NOTIFICATION_EVENT_ID_KEY_NAME,
>             Long.toString(event.getEventId()));
>       }
>   }
> {code}
> the above code in DBNotificationListner having the object lock wont be guarantee enough
to make sure that all events get a unique id. The transaction isolation level at the db "read-comitted"
or "repeatable-read"  would  also not guarantee the same, unless a lock is at the db level
preferably on table {{NOTIFICATION_SEQUENCE}} which only has one row.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message