hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elliot West (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-11228) Mutation API should uses semi-shared locks.
Date Fri, 10 Jul 2015 14:00:08 GMT
Elliot West created HIVE-11228:
----------------------------------

             Summary: Mutation API should uses semi-shared locks.
                 Key: HIVE-11228
                 URL: https://issues.apache.org/jira/browse/HIVE-11228
             Project: Hive
          Issue Type: Bug
          Components: HCatalog
    Affects Versions: 2.0.0
            Reporter: Elliot West
            Assignee: Elliot West


An [issue was identified|https://issues.apache.org/jira/browse/HIVE-10165?focusedCommentId=14619640&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14619640]
by [~ekoifman] in the streaming mutation API (HIVE-10165) where an insufficiently restrictive
lock was being used when issuing updates and deletes to ACID tables and partitions. A shared
lock was being used where in fact a semi-shared lock is required. Additionally, the current
lock scope targets the entire table, whereas in theory if the table is partitioned, then only
the affected partitions are required to participate in the semi-shared lock. However, there
are a couple of technical challenges that prevent the locks currently being applied on a per-partition
basis:

* It is expected that the affected partitions are not known in advance so individual partition
locks would need to be acquired as needed.
* The API is expected to execute in a clustered environment and so acquiring these locks as
on an ‘as needed’ basis presents a risk that the meta store may become overwhelmed. This
is expected to be less of an problem when an HBase based meta store is introduced (HIVE-9452).
* My understanding is that multiple fine grained lock acquisitions for a single transaction
are not possible at present. When they are available they’ll introduce the possibility that
deadlocks can occur. This should be better handled when HIVE-9675 is complete.

Therefore, as advised, at this time the system will obtain a semi-shared lock on participating
tables. Although this will prevent other concurrent writes, it will preserve snapshot isolation
when reading.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message