hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ferdinand Xu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8136) Reduce table locking
Date Thu, 29 Jan 2015 08:58:35 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14296607#comment-14296607
] 

Ferdinand Xu commented on HIVE-8136:
------------------------------------

Currently the following alter table write type is trying to acquire an exclusive lock.
DDL_EXCLUSIVE;
RENAMECOLUMN
ADDCLUSTERSORTCOLUMN:
ADDFILEFORMAT:
DROPPROPS:
REPLACECOLS:
ARCHIVE:
UNARCHIVE:
ALTERPROTECTMODE:
ALTERPARTITIONPROTECTMODE:
ALTERLOCATION:
DROPPARTITION:
RENAMEPARTITION:
ADDSKEWEDBY:
ALTERSKEWEDLOCATION:
ALTERBUCKETNUM:
ALTERPARTITION:
ADDCOLS:
RENAME:
TRUNCATE:
MERGEFILES:

Other following is using shared lock:
  ADDSERDE
  ADDPARTITION
  ADDSERDEPROPS
  ADDPROPS

Others has no lock:
  COMPACT
  TOUCH

For changing table structure, an exclusive lock is a must. Most of the cases use the exclusive
lock since it changes the table or partition structure currently. For adding cluster column
and sort column, we can use shared lock for the following reason.
{quote}
The CLUSTERED BY and SORTED BY creation commands do not affect how data is inserted into a
table – only how it is read. This means that users must be careful to insert data correctly
by specifying the number of reducers to be equal to the number of buckets, and using CLUSTER
BY and SORT BY commands in their query.
{quote}
For changing the properties, I think we can use no lock if it doesn't change the structure
of the table. We can do a follow-up jira.  Any thought about it, [~brocknoland]?

> Reduce table locking
> --------------------
>
>                 Key: HIVE-8136
>                 URL: https://issues.apache.org/jira/browse/HIVE-8136
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Brock Noland
>            Assignee: Ferdinand Xu
>
> When using ZK for concurrency control, some statements require an exclusive table lock
when they are atomic. Such as setting a tables location.
> This JIRA is to analyze the scope of statements like ALTER TABLE and see if we can reduce
the locking required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message