hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chaoyu Tang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13959) MoveTask should only release its query associated locks
Date Wed, 08 Jun 2016 00:18:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15319761#comment-15319761
] 

Chaoyu Tang commented on HIVE-13959:
------------------------------------

[~ychena] Thanks for the review. For your analysis and questions, please see below:
Yes -- one WriteEntity map to one List<HiveLockObj>
Yes -- These list of HiveLockObj are all created during acquireLocks related to the query.
Yes -- In the releaseLocks code, lockObj.getObj() return HiveLockObject
The problem is here: List<HiveLock> locks = lockMgr.getLocks(lockObj.getObj(), false,
true); it returns all locks under the pathName which might not related to this MoveTask query:
{code} 
The getLocks method in ZookeeperHiveLockManager:
private static List<HiveLock> getLocks(HiveConf conf,
      HiveLockObject key, String parent, boolean verifyTablePartition, boolean fetchData)
      throws LockException {
    List<HiveLock> locks = new ArrayList<HiveLock>();
    List<String> children;
    boolean recurse = true;
    String commonParent;

    try {
      if (key != null) {
        commonParent = "/" + parent + "/" + key.getName();
        children = curatorFramework.getChildren().forPath(commonParent); 
        /* ==> this call returns all locks under commonParent, say db/cdhpart/LOCK-SHARE-000000,
db/cdhpart/LOCK-SHARE-000001 for pathNames db/cdhpart */
        recurse = false;
      }
      else {
        commonParent = "/" + parent;
        children = curatorFramework.getChildren().forPath(commonParent);
      }
    } catch (Exception e) {
      // no locks present
      return locks;
    }
{code}
For an example, if we run query1 in one session "insert overwrite table cdhpart partition
(level1= 'l1', level2='l2', level3 = 'l3', level4) select key, value, level4 from cdhsrc;"
and query2 in the other session concurrently "select * from cdhpart where level1 = 'l1'"
query1 and query2 both have its own znode (lock) under pathNames (db/cdhpart/) say LOCK-SHARE-000000,
LOCK-SHARE-000001 respectively, the getLocks for HiveLockObject key with its getName() value
db/cdhpart/ will return both LOCK-SHARE-000000, LOCK-SHARE-000001. But LOCK-SHARE-000001 is
not in the ctx.getHiveLocks(), the lock list for the query1, so ctx.getHiveLocks().remove()
returns false because the HiveLockObjectData.equals always return false due to the different
queryStr/queryId, therefore lockMgr.unlock(lock) should not be called to unlock the LOCK-SHARE-000001
for query2.



> MoveTask should only release its query associated locks
> -------------------------------------------------------
>
>                 Key: HIVE-13959
>                 URL: https://issues.apache.org/jira/browse/HIVE-13959
>             Project: Hive
>          Issue Type: Bug
>          Components: Locking
>            Reporter: Chaoyu Tang
>            Assignee: Chaoyu Tang
>         Attachments: HIVE-13959.patch, HIVE-13959.patch
>
>
> releaseLocks in MoveTask releases all locks under a HiveLockObject pathNames. But some
of locks under this pathNames might be for other queries and should not be released.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message