Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of jdcryans@gmail.com designates
 209.85.216.169 as permitted sender)
MIME-Version: 1.0
Sender: jdcryans@gmail.com
In-Reply-To: <CCE3493E6FDC454D806BB4F1544BE57D@gallotta.co.za>
References: <CCE3493E6FDC454D806BB4F1544BE57D@gallotta.co.za>
Date: Tue, 24 Jul 2012 14:58:38 -0700
Message-ID: 
 <CAGpTDNe7s+Vpk9f3xOev=VtMee=pROUdxKZmNXvzfkEy1BCibQ@mail.gmail.com>
Subject: Re: UnknownRowLockException
From: Jean-Daniel Cryans <jdcryans@apache.org>
To: user@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Row locks don't move with regions or splits... are such things
happening frequently?

Also, any reason to not use HBase's own increment method that's much
more efficient?

http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#=
incrementColumnValue(byte[],
byte[], byte[], long)

J-D

On Tue, Jul 24, 2012 at 2:53 PM, Marco Gallotta <marco@gallotta.co.za> wrot=
e:
> Hi there
>
> I am getting an UnknownRowLockException when adding locks to the incremen=
t() function below. The full stack trace is at the end of this message.
>
> There are a few other places I am acquiring locks, but I only ever acquir=
e a single lock for one piece of code and the rest go through fine. It's on=
ly when I enable the locks in this function that I get the exception. It's =
fairly simple, and I can't find anything obviously wrong with it so my best=
 guess is there's some quirk with hbase locks that I'm unaware of. Any idea=
s?
>
> One thing to note is that I am calling increment() a couple times consecu=
tively for the same row.
>
> Marco
>
>   public static void increment(String tableName, String key, String famil=
y,
>                          String qualifier, int value, boolean useShort,
>                          Configuration config)
>       throws IOException{
>     HTable table =3D new HTable(config,
>         config.get("app", "geomon-test") + "." + tableName);
>     Get get =3D new Get(key.getBytes());
>     get.addColumn(family.getBytes(),
>         qualifier.getBytes());
>     int before =3D 0;
>     RowLock lock =3D table.lockRow(key.getBytes());
>     try {
>       Result result =3D table.get(get);
>       if (!result.isEmpty()) {
>         if (useShort) {
>           before =3D (int) Bytes.toShort(result.getValue(family.getBytes(=
),
>               qualifier.getBytes()));
>         } else {
>           before =3D Bytes.toInt(result.getValue(family.getBytes(),
>               qualifier.getBytes()));
>         }
>       }
>       if (useShort) {
>         value =3D Math.min(before + value, Short.MAX_VALUE);
>       } else {
>         value =3D (int) Math.min((long) before + value, Integer.MAX_VALUE=
);
>       }
>
>       Put put =3D new Put(key.getBytes());
>       put.add(family.getBytes(),
>           qualifier.getBytes(),
>           Bytes.toBytes(useShort ? (short) value : value));
>       table.put(put);
>     } catch (Exception e) {
>       System.err.println(e.getMessage());
>     } finally {
>       table.unlockRow(lock);
>     }
>   }
>
>
> 12/07/24 14:20:58 INFO mapred.JobClient: Task Id : attempt_201207241320_0=
002_r_000000_2, Status : FAILEDorg.apache.hadoop.hbase.UnknownRowLockExcept=
ion: org.apache.hadoop.hbase.UnknownRowLockException: -3110061788478328763
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native =
Method)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeCo=
nstructorAccessorImpl.java:57)        at sun.reflect.DelegatingConstructorA=
ccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)        a=
t java.lang.reflect.Constructor.newInstance(Constructor.java:532)
>         at org.apache.hadoop.ipc.RemoteException.instantiateException(Rem=
oteException.java:95)
>         at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(Re=
moteException.java:79)
>         at org.apache.hadoop.hbase.client.ServerCallable.translateExcepti=
on(ServerCallable.java:228)
>         at org.apache.hadoop.hbase.client.ServerCallable.withRetries(Serv=
erCallable.java:166)
>         at org.apache.hadoop.hbase.client.HTable.unlockRow(HTable.java:10=
31)
>         at Util.increment(Util.java:170)
>         at Aggregate$EventReducer.processUserEvent(Aggregate.java:161)
>         at Aggregate$EventReducer.processUser(Aggregate.java:119)
>         at Aggregate$EventReducer.reduce(Aggregate.java:189)
>         at Aggregate$EventReducer.reduce(Aggregate.java:78)
>         at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
>         at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.j=
ava:649)
>         at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:416)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroup=
Information.java:1121)
>         at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase=
.UnknownRowLockException: -3110061788478328763
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.unlockRow(H=
RegionServer.java:2633)
>         at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth=
odAccessorImpl.java:43)
>
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(Writ=
ableRpcEngine.java:364)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServe=
r.java:1376)
>
>         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:=
918)
>         at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(W=
ritableRpcEngine.java:150)
>         at $Proxy3.unlockRow(Unknown Source)
>         at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1033=
)
>         at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1031=
)
>         at org.apache.hadoop.hbase.client.ServerCallable.withRetries(Serv=
erCallable.java:163)
>         ... 14 more
>
>
>
> --
> Marco Gallotta | Mountain View, California
> Software Engineer, Infrastructure | Loki Studios
> fb.me/marco.gallotta | twitter.com/marcog
> marco@gallotta.co.za | +1 (650) 417-3313
>
> Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
>