hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marco Gallotta <ma...@gallotta.co.za>
Subject Re: UnknownRowLockException
Date Tue, 24 Jul 2012 22:08:45 GMT
I was unaware of locks not moving with regions/splits. Hmm…I just came across http://jerryjcw.blogspot.com/2009/10/hbase-notes-casual-remark-about-row.html
which I'm going to try.

HBase's increment method says "readers do not take row locks so get and scan operations can
see this operation partially completed", which could cause problems.

Marco  

--  
Marco Gallotta | Mountain View, California
Software Engineer, Infrastructure | Loki Studios
fb.me/marco.gallotta | twitter.com/marcog
marco@gallotta.co.za | +1 (650) 417-3313

Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Tuesday 24 July 2012 at 2:58 PM, Jean-Daniel Cryans wrote:

> Row locks don't move with regions or splits... are such things
> happening frequently?
>  
> Also, any reason to not use HBase's own increment method that's much
> more efficient?
>  
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#incrementColumnValue(byte[],
> byte[], byte[], long)
>  
> J-D
>  
> On Tue, Jul 24, 2012 at 2:53 PM, Marco Gallotta <marco@gallotta.co.za (mailto:marco@gallotta.co.za)>
wrote:
> > Hi there
> >  
> > I am getting an UnknownRowLockException when adding locks to the increment() function
below. The full stack trace is at the end of this message.
> >  
> > There are a few other places I am acquiring locks, but I only ever acquire a single
lock for one piece of code and the rest go through fine. It's only when I enable the locks
in this function that I get the exception. It's fairly simple, and I can't find anything obviously
wrong with it so my best guess is there's some quirk with hbase locks that I'm unaware of.
Any ideas?
> >  
> > One thing to note is that I am calling increment() a couple times consecutively
for the same row.
> >  
> > Marco
> >  
> > public static void increment(String tableName, String key, String family,
> > String qualifier, int value, boolean useShort,
> > Configuration config)
> > throws IOException{
> > HTable table = new HTable(config,
> > config.get("app", "geomon-test") + "." + tableName);
> > Get get = new Get(key.getBytes());
> > get.addColumn(family.getBytes(),
> > qualifier.getBytes());
> > int before = 0;
> > RowLock lock = table.lockRow(key.getBytes());
> > try {
> > Result result = table.get(get);
> > if (!result.isEmpty()) {
> > if (useShort) {
> > before = (int) Bytes.toShort(result.getValue(family.getBytes(),
> > qualifier.getBytes()));
> > } else {
> > before = Bytes.toInt(result.getValue(family.getBytes(),
> > qualifier.getBytes()));
> > }
> > }
> > if (useShort) {
> > value = Math.min(before + value, Short.MAX_VALUE);
> > } else {
> > value = (int) Math.min((long) before + value, Integer.MAX_VALUE);
> > }
> >  
> > Put put = new Put(key.getBytes());
> > put.add(family.getBytes(),
> > qualifier.getBytes(),
> > Bytes.toBytes(useShort ? (short) value : value));
> > table.put(put);
> > } catch (Exception e) {
> > System.err.println(e.getMessage());
> > } finally {
> > table.unlockRow(lock);
> > }
> > }
> >  
> >  
> > 12/07/24 14:20:58 INFO mapred.JobClient: Task Id : attempt_201207241320_0002_r_000000_2,
Status : FAILEDorg.apache.hadoop.hbase.UnknownRowLockException: org.apache.hadoop.hbase.UnknownRowLockException:
-3110061788478328763
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
> > at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
> > at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
> > at org.apache.hadoop.hbase.client.ServerCallable.translateException(ServerCallable.java:228)
> > at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:166)
> > at org.apache.hadoop.hbase.client.HTable.unlockRow(HTable.java:1031)
> > at Util.increment(Util.java:170)
> > at Aggregate$EventReducer.processUserEvent(Aggregate.java:161)
> > at Aggregate$EventReducer.processUser(Aggregate.java:119)
> > at Aggregate$EventReducer.reduce(Aggregate.java:189)
> > at Aggregate$EventReducer.reduce(Aggregate.java:78)
> > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> > at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
> > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
> > at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:416)
> > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> > at org.apache.hadoop.mapred.Child.main(Child.java:249)
> > Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.UnknownRowLockException:
-3110061788478328763
> > at org.apache.hadoop.hbase.regionserver.HRegionServer.unlockRow(HRegionServer.java:2633)
> > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
> > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >  
> > at java.lang.reflect.Method.invoke(Method.java:616)
> > at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
> > at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1376)
> >  
> > at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:918)
> > at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
> > at $Proxy3.unlockRow(Unknown Source)
> > at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1033)
> > at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1031)
> > at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:163)
> > ... 14 more
> >  
> >  
> >  
> > --
> > Marco Gallotta | Mountain View, California
> > Software Engineer, Infrastructure | Loki Studios
> > fb.me/marco.gallotta (http://fb.me/marco.gallotta) | twitter.com/marcog (http://twitter.com/marcog)
> > marco@gallotta.co.za (mailto:marco@gallotta.co.za) | +1 (650) 417-3313
> >  
> > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)  


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message