hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11228) Mutation API should use semi-shared locks.
Date Fri, 10 Jul 2015 23:23:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623052#comment-14623052
] 

Eugene Koifman commented on HIVE-11228:
---------------------------------------

+1 

> Mutation API should use semi-shared locks.
> ------------------------------------------
>
>                 Key: HIVE-11228
>                 URL: https://issues.apache.org/jira/browse/HIVE-11228
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog
>    Affects Versions: 2.0.0
>            Reporter: Elliot West
>            Assignee: Elliot West
>              Labels: streaming_api, transaction, transactions
>         Attachments: HIVE-11228.1.patch
>
>
> An [issue was identified|https://issues.apache.org/jira/browse/HIVE-10165?focusedCommentId=14619640&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14619640]
by [~ekoifman] in the streaming mutation API (HIVE-10165) where an insufficiently restrictive
lock was being used when issuing updates and deletes to ACID tables and partitions. A shared
lock was being used where in fact a semi-shared lock is required. Additionally, the current
lock scope targets the entire table, whereas in theory if the table is partitioned, then only
the affected partitions are required to participate in the semi-shared lock. However, there
are a couple of technical challenges that prevent the locks currently being applied on a per-partition
basis:
> * It is expected that the affected partitions are not known in advance so individual
partition locks would need to be acquired as needed.
> * The API is expected to execute in a clustered environment and so acquiring these locks
as on an ‘as needed’ basis presents a risk that the meta store may become overwhelmed.
This is expected to be less of an problem when an HBase based meta store is introduced (HIVE-9452).
> * My understanding is that multiple fine grained lock acquisitions for a single transaction
are not possible at present. When they are available they’ll introduce the possibility that
deadlocks can occur. This should be better handled when HIVE-9675 is complete.
> Therefore, as advised, at this time the system will obtain a semi-shared lock on participating
tables. Although this will prevent other concurrent writes, it will preserve snapshot isolation
when reading.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message