hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stack <st...@duboce.net>
Subject Re: Problems when executing many (?) HTable.lockRow()
Date Sat, 09 May 2009 02:14:12 GMT
You've seen
http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/HTable.html#checkAndSave(org.apache.hadoop.hbase.io.BatchUpdate,%20org.apache.hadoop.hbase.io.HbaseMapWritable,%20org.apache.hadoop.hbase.client.RowLock)
?

What do you think is going on in your slow test Guilherme?  Is it just
thread scheduler not being fair?  Do all attempts take same amount of time?
Or could it be that the regionserver only puts up ten listeners and somehow
when you have 20 threads there is a sort of deadlock going on?

St.Ack

On Fri, May 8, 2009 at 6:57 PM, Guilherme Germoglio <germoglio@gmail.com>wrote:

> thank you, Ryan.
>
> what about changing a row value to something that depends on its previous
> value? An atomic increment of an integer, for example. I'm thinking
> something like the following (in pseudocode - sort of :-):
>
> lock = htable.lockRow(row);
> value = htable.getRow(row, lock);
> value = doSomething(value);
> batchupdate.put(row, column, value);
> htable.commit(batchupdate, lock);
> htable.unlockRow(lock);
>
> On Fri, May 8, 2009 at 8:51 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
>
> > Hey,
> >
> > You can bundle any number of operations (put and delete) to 1 row with
> > BatchUpdate thus obviating the need for using explicit row locks (the
> > server
> > does the locks internally as well, so in this case you are locking
> twice).
> >
> > -ryan
> >
> > On Fri, May 8, 2009 at 4:29 PM, Guilherme Germoglio <germoglio@gmail.com
> > >wrote:
> >
> > > Hello,
> > >
> > > I've been running some tests using HTable.lockRow but I think it is not
> > > performing well. One of my tests is quite simple: run several threads
> > that
> > > will execute the following lines for the same row:
> > >
> > >               RowLock rowLock = htable.lockRow(Bytes.toBytes(row));
> > >               htable.commit(batchupdate, rowLock);
> > >               htable.unlockRow(rowLock);
> > >
> > > where the batchupdate only does a single "put" on a single column.
> > >
> > > The problem is that it is running very very slow when I use only 20
> > > threads:
> > > about 30 minutes against the 6~10 seconds of running with 10 threads.
> > Also,
> > > when using 20 threads, some *java.io.IOException: java.io.IOException:
> > > Invalid row lock* messages are printed.
> > >
> > > Since it is the first time I'm using RowLocks, I'm not sure if the
> > problem
> > > is in my test or in HBase. So, I'm asking you for help.
> > >
> > > I'm using hbase 0.19.2 (rc) in pseudo-distributed mode on a mac book,
> but
> > > the behavior is the same when using 0.19.0. Logs, hbase-site.xml (which
> > is
> > > the default) and the test can be found on the following link:
> > >
> > > http://germoglio.googlepages.com/logs.zip
> > >
> > > Please notice that although this test doesn't make much sense of
> locking,
> > > the idea is to evolve it in order to perform a few operations besides
> the
> > > only single put between lockRow and unlockRow methods.
> > >
> > > Thank you very much,
> > >
> > > --
> > > Guilherme
> > >
> > > msn: guigermoglio@hotmail.com
> > > homepage: http://germoglio.googlepages.com
> > >
> >
>
>
>
> --
> Guilherme
>
> msn: guigermoglio@hotmail.com
> homepage: http://germoglio.googlepages.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message