hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hive/Locking" by NamitJain
Date Wed, 05 May 2010 05:53:25 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Hive/Locking" page has been changed by NamitJain.
http://wiki.apache.org/hadoop/Hive/Locking?action=diff&rev1=13&rev2=14

--------------------------------------------------

  
  The rational behind the lock mode to acquire is as follows:
  
- For a non-partitioned table, the lock modes are pretty intutive. When the table is being
read, a S
+ For a non-partitioned table, the lock modes are pretty intuitive. When the table is being
read, a S
  lock is	acquired, whereas an X lock is acquired	for all	other operations (insert into the
table,
  alter table of any kind	etc.)
  
@@ -50, +50 @@

  ||'''alter table T1 add partition P1'''||'''S on T1, X on T1.P1'''||
  ||'''alter table T1 drop partition P1'''||'''S on T1, X on T1.P1'''||
  ||'''alter table T1 touch partition P1'''||'''S on T1, X on T1.P1'''||
- ||'''alter table T1 set serdeProperties '''||'''S on T1'''||
+ ||'''alter table T1 set serdeproperties '''||'''S on T1'''||
  ||'''alter table T1 set serializer '''||'''S on T1'''||
  ||'''alter table T1 set file format '''||'''S on T1'''||
  ||'''alter table T1 set tblproperties  '''||'''X on T1'''||
@@ -63, +63 @@

  number of partitions may not be known, an exclusive lock is taken on the table, or the prefix
that 
  is known. 
  
+ The lock modes are hierarchical in nature - if 'S' lock is acquired on T, implicitly 'S'
lock is acquired
+ on all partitions of T.
+ 
  Two new configurable parameters will be added to decide the number of retries for the lock
and the 
  wait time between each retry. If the number of retries are really high, it can lead to a
live lock.
  Look at ZooKeeper (http://hadoop.apache.org/zookeeper/docs/r3.1.2/recipes.html#sc_recipes_Locks)
to
@@ -70, +73 @@

  the lock request will be denied. The existing locks will be released, and all of them will
be retried
  after the retry interval.
  
+ The recipe listed above will not work as specified, because of the hierarchical nature of
locks.
+ The 'S' lock for table T are specified as follows:
+  * Call create( ) to create a node with pathname "/warehouse/T/read-". This is the lock
node use later in the protocol. Make sure to set ephemeral flag.
+  * Call getChildren( ) on the lock node without setting the watch flag.
+  * If there are a child with a pathname starting with "write-", then the lock cannot be
acquired. Delete the node created in the first step and return.
+  * For all sub-directories of T, if there is a child with a pathname starting with "write-",
the lock cannot be acquired. Delete the node created in the first step and return.
+  * Otherwise the lock is granted.
+ 
+ The 'X' lock for table T are specified as follows:
+  * Call create( ) to create a node with pathname "/warehouse/T/write-". This is the lock
node use later in the protocol. Make sure to set the ephemeral flag.
+  * Call getChildren( ) on the lock node without setting the watch flag.
+  * If there are a child with a pathname starting with "read-", then the lock cannot be acquired.
Delete the node created in the first step and return.
+  * For all sub-directories of T, if there is a child with a pathname starting with "read-"
or "write-", the lock cannot be acquired. Delete the node created in the first step and return.
+  * Otherwise the lock is granted.
+ 
+ The proposed scheme starves the writers for readers. In case of long readers, it may lead
to starvation for writers.
+ 
  The default hive behavior will not be changed, and concurrency will not be supported.
  

Mime
View raw message