hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nachiket Vaidya (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13966) DbNotificationListener: can loose DDL operation notifications
Date Wed, 08 Jun 2016 18:01:21 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15321083#comment-15321083
] 

Nachiket Vaidya commented on HIVE-13966:
----------------------------------------

+ When operation fails, should we continue adding an entry to notification log or not?
>> This is fine as this is false positive and one can skip further operation if the
given entity is not present.

+ When operation is correct, but notification log fails, should we just display a warning
message? or rollback the operation?
>> This can loose the information that something is changed in metadata. Ideally we
should rollback the operation. But it can be expensive. A simple fix is to add to notification
log before operation. But this will then applicable for all metastore listeners. I do not
have good solution/suggestion for the same.
Actually we will get error when operation is successful but adding to notification log fails.
But if application chose to ignore errors, this inconsistency can create problem.
What is the contract of listeners? They do not run in the same transaction as that of operation?
and they just notify that operation was executed irrespective of the result of the operation?

> DbNotificationListener: can loose DDL operation notifications
> -------------------------------------------------------------
>
>                 Key: HIVE-13966
>                 URL: https://issues.apache.org/jira/browse/HIVE-13966
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog
>            Reporter: Nachiket Vaidya
>            Priority: Critical
>
> The code for each API in HiveMetaStore.java is like this:
> 1. openTransaction()
> 2. -- operation--
> 3. commit() or rollback() based on result of the operation.
> 4. add entry to notification log (unconditionally)
> If the operation is failed (in step 2), we still add entry to notification log. Found
this issue in testing.
> It is still ok as this is the case of false positive.
> If the operation is successful and adding to notification log failed, the user will get
an MetaException. It will not rollback the operation, as it is already committed. We need
to handle this case so that we will not have false negatives.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message