Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Date: Tue, 24 Jul 2012 15:08:45 -0700
From: Marco Gallotta <marco@gallotta.co.za>
To: user@hbase.apache.org
Message-ID: <786FF04E80AD4CF2A8FAE5A42AAAE699@gallotta.co.za>
In-Reply-To: 
 <CAGpTDNe7s+Vpk9f3xOev=VtMee=pROUdxKZmNXvzfkEy1BCibQ@mail.gmail.com>
References: <CCE3493E6FDC454D806BB4F1544BE57D@gallotta.co.za>
 <CAGpTDNe7s+Vpk9f3xOev=VtMee=pROUdxKZmNXvzfkEy1BCibQ@mail.gmail.com>
Subject: Re: UnknownRowLockException
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="500f1ced_4e647fe4_a019"

--500f1ced_4e647fe4_a019
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

I was unaware of locks not moving with regions/splits. Hmm=E2=80=A6I just=
 came across http://jerryjcw.blogspot.com/2009/10/hbase-notes-casual-rema=
rk-about-row.html which I'm going to try.

HBase's increment method says =22readers do not take row locks so get and=
 scan operations can see this operation partially completed=22, which cou=
ld cause problems.

Marco =20

-- =20
Marco Gallotta =7C Mountain View, California
Software Engineer, Infrastructure =7C Loki Studios
fb.me/marco.gallotta =7C twitter.com/marcog
marco=40gallotta.co.za =7C +1 (650) 417-3313

Sent with Sparrow (http://www.sparrowmailapp.com/=3Fsig)


On Tuesday 24 July 2012 at 2:58 PM, Jean-Daniel Cryans wrote:

> Row locks don't move with regions or splits... are such things
> happening frequently=3F
> =20
> Also, any reason to not use HBase's own increment method that's much
> more efficient=3F
> =20
> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.h=
tml=23incrementColumnValue(byte=5B=5D,
> byte=5B=5D, byte=5B=5D, long)
> =20
> J-D
> =20
> On Tue, Jul 24, 2012 at 2:53 PM, Marco Gallotta <marco=40gallotta.co.za=
 (mailto:marco=40gallotta.co.za)> wrote:
> > Hi there
> > =20
> > I am getting an UnknownRowLockException when adding locks to the incr=
ement() function below. The full stack trace is at the end of this messag=
e.
> > =20
> > There are a few other places I am acquiring locks, but I only ever ac=
quire a single lock for one piece of code and the rest go through fine. I=
t's only when I enable the locks in this function that I get the exceptio=
n. It's fairly simple, and I can't find anything obviously wrong with it =
so my best guess is there's some quirk with hbase locks that I'm unaware =
of. Any ideas=3F
> > =20
> > One thing to note is that I am calling increment() a couple times con=
secutively for the same row.
> > =20
> > Marco
> > =20
> > public static void increment(String tableName, String key, String fam=
ily,
> > String qualifier, int value, boolean useShort,
> > Configuration config)
> > throws IOException=7B
> > HTable table =3D new HTable(config,
> > config.get(=22app=22, =22geomon-test=22) + =22.=22 + tableName);
> > Get get =3D new Get(key.getBytes());
> > get.addColumn(family.getBytes(),
> > qualifier.getBytes());
> > int before =3D 0;
> > RowLock lock =3D table.lockRow(key.getBytes());
> > try =7B
> > Result result =3D table.get(get);
> > if (=21result.isEmpty()) =7B
> > if (useShort) =7B
> > before =3D (int) Bytes.toShort(result.getValue(family.getBytes(),
> > qualifier.getBytes()));
> > =7D else =7B
> > before =3D Bytes.toInt(result.getValue(family.getBytes(),
> > qualifier.getBytes()));
> > =7D
> > =7D
> > if (useShort) =7B
> > value =3D Math.min(before + value, Short.MAX=5FVALUE);
> > =7D else =7B
> > value =3D (int) Math.min((long) before + value, Integer.MAX=5FVALUE);=

> > =7D
> > =20
> > Put put =3D new Put(key.getBytes());
> > put.add(family.getBytes(),
> > qualifier.getBytes(),
> > Bytes.toBytes(useShort =3F (short) value : value));
> > table.put(put);
> > =7D catch (Exception e) =7B
> > System.err.println(e.getMessage());
> > =7D finally =7B
> > table.unlockRow(lock);
> > =7D
> > =7D
> > =20
> > =20
> > 12/07/24 14:20:58 IN=46O mapred.JobClient: Task Id : attempt=5F201207=
241320=5F0002=5Fr=5F000000=5F2, Status : =46AILEDorg.apache.hadoop.hbase.=
UnknownRowLockException: org.apache.hadoop.hbase.UnknownRowLockException:=
 -3110061788478328763
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Meth=
od)
> > at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstr=
uctorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorIm=
pl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.re=
flect.Constructor.newInstance(Constructor.java:532)
> > at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteE=
xception.java:95)
> > at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(Remote=
Exception.java:79)
> > at org.apache.hadoop.hbase.client.ServerCallable.translateException(S=
erverCallable.java:228)
> > at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCa=
llable.java:166)
> > at org.apache.hadoop.hbase.client.HTable.unlockRow(HTable.java:1031)
> > at Util.increment(Util.java:170)
> > at Aggregate=24EventReducer.processUserEvent(Aggregate.java:161)
> > at Aggregate=24EventReducer.processUser(Aggregate.java:119)
> > at Aggregate=24EventReducer.reduce(Aggregate.java:189)
> > at Aggregate=24EventReducer.reduce(Aggregate.java:78)
> > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
> > at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:=
649)
> > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
> > at org.apache.hadoop.mapred.Child=244.run(Child.java:255)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:416)
> > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInfo=
rmation.java:1121)
> > at org.apache.hadoop.mapred.Child.main(Child.java:249)
> > Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.h=
base.UnknownRowLockException: -3110061788478328763
> > at org.apache.hadoop.hbase.regionserver.HRegionServer.unlockRow(HRegi=
onServer.java:2633)
> > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
> > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAc=
cessorImpl.java:43)
> > =20
> > at java.lang.reflect.Method.invoke(Method.java:616)
> > at org.apache.hadoop.hbase.ipc.WritableRpcEngine=24Server.call(Writab=
leRpcEngine.java:364)
> > at org.apache.hadoop.hbase.ipc.HBaseServer=24Handler.run(HBaseServer.=
java:1376)
> > =20
> > at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:918)=

> > at org.apache.hadoop.hbase.ipc.WritableRpcEngine=24Invoker.invoke(Wri=
tableRpcEngine.java:150)
> > at =24Proxy3.unlockRow(Unknown Source)
> > at org.apache.hadoop.hbase.client.HTable=2415.call(HTable.java:1033)
> > at org.apache.hadoop.hbase.client.HTable=2415.call(HTable.java:1031)
> > at org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCa=
llable.java:163)
> > ... 14 more
> > =20
> > =20
> > =20
> > --
> > Marco Gallotta =7C Mountain View, California
> > Software Engineer, Infrastructure =7C Loki Studios
> > fb.me/marco.gallotta (http://fb.me/marco.gallotta) =7C twitter.com/ma=
rcog (http://twitter.com/marcog)
> > marco=40gallotta.co.za (mailto:marco=40gallotta.co.za) =7C +1 (650) 4=
17-3313
> > =20
> > Sent with Sparrow (http://www.sparrowmailapp.com/=3Fsig) =20


--500f1ced_4e647fe4_a019--