hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hive/Locking" by NamitJain
Date Wed, 05 May 2010 01:14:16 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Hive/Locking" page has been changed by NamitJain.


  = Use Cases =
  Concurrency support (http://issues.apache.org/jira/browse/HIVE-1293) is a must in databases
and their usecases are well understood. At a minimum, we want to support concurrent readers
and writers whenever possible.
- It would be useful to add a mechanism to discover the current locks which have been acquired.
+ It would be useful to add a mechanism to discover the current locks which have been acquired.
There is no immediate requirement to add an API to explicitly acquire any locks, so all locks
would be acquired implicitly.
+ he following lock modes are defined in hive (Note that Intent lock is not needed).
+ Shared    (S)
+ Exclusive (X)
+ As the name suggests, multiple shared locks can be acquired at the same time, whereas X
lock blocks all
+ other locks.
+ The compatiblity matrix is as follows:
+ For some operations, locks are hierarchial in nature - for eg. for some partition operations,
the table
+ is also locked (for eg. to make sure that the table cannot be dropped while a new partition
is being created)
+ The rational behind the lock mode to acquire is as follows:
+ For a non-partitioned table, the lock modes are pretty intutive. When the table is being
read, a S
+ lock is	acquired, whereas an X lock is acquired	for all	other operations (insert into the
+ alter table of any kind	etc.)
+ For a partitioned table, the idea is as	follows:
+ A 'S' lock on table and relevant partition is acquired when a read is being performed.
+ For all	other operations, an 'X' lock is taken on the partition. However, if the change
+ only applicable	to the newer partitions, a 'S' lock is acquired	on the table, whereas if
+ change is applicable to	all partitions,	a 'X' lock is acquired on the table. Thus, older
+ partitions can be read and written into, while the newer partitions are	being converted
+ RCFile.
+ Based on this, the lock	acquired for an	operation is as	follows:
+ In order to avoid deadlocks, a very simple scheme is proposed here. All	the objects to be
+ are sorted lexicographically, and the required mode lock is acquired. Note that	in some
cases, the
+ list of objects may not be known - for eg. in case of dynamic partitions, the list of partitions
+ being modified is not known at compile time - so, the list is generated conservatively.
  = Syntax =

View raw message