hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guilherme Germoglio <germog...@gmail.com>
Subject Re: Problems when executing many (?) HTable.lockRow()
Date Sun, 10 May 2009 17:19:33 GMT
On Fri, May 8, 2009 at 11:14 PM, stack <stack@duboce.net> wrote:

> You've seen
>
> http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/HTable.html#checkAndSave(org.apache.hadoop.hbase.io.BatchUpdate,%20org.apache.hadoop.hbase.io.HbaseMapWritable,%20org.apache.hadoop.hbase.client.RowLock)<http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/client/HTable.html#checkAndSave%28org.apache.hadoop.hbase.io.BatchUpdate,%20org.apache.hadoop.hbase.io.HbaseMapWritable,%20org.apache.hadoop.hbase.client.RowLock%29>
> ?
>

Hi Stack,

Is the rowlock parameter in checkAndSave mandatory? Or can it be null? If it
can't be null, I'll probably have the problem of many lockRow() calls as
well.


>
> What do you think is going on in your slow test Guilherme?  Is it just
> thread scheduler not being fair?  Do all attempts take same amount of time?
> Or could it be that the regionserver only puts up ten listeners and somehow
> when you have 20 threads there is a sort of deadlock going on?
>
> St.Ack
>
> On Fri, May 8, 2009 at 6:57 PM, Guilherme Germoglio <germoglio@gmail.com
> >wrote:
>
> > thank you, Ryan.
> >
> > what about changing a row value to something that depends on its previous
> > value? An atomic increment of an integer, for example. I'm thinking
> > something like the following (in pseudocode - sort of :-):
> >
> > lock = htable.lockRow(row);
> > value = htable.getRow(row, lock);
> > value = doSomething(value);
> > batchupdate.put(row, column, value);
> > htable.commit(batchupdate, lock);
> > htable.unlockRow(lock);
> >
> > On Fri, May 8, 2009 at 8:51 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
> >
> > > Hey,
> > >
> > > You can bundle any number of operations (put and delete) to 1 row with
> > > BatchUpdate thus obviating the need for using explicit row locks (the
> > > server
> > > does the locks internally as well, so in this case you are locking
> > twice).
> > >
> > > -ryan
> > >
> > > On Fri, May 8, 2009 at 4:29 PM, Guilherme Germoglio <
> germoglio@gmail.com
> > > >wrote:
> > >
> > > > Hello,
> > > >
> > > > I've been running some tests using HTable.lockRow but I think it is
> not
> > > > performing well. One of my tests is quite simple: run several threads
> > > that
> > > > will execute the following lines for the same row:
> > > >
> > > >               RowLock rowLock = htable.lockRow(Bytes.toBytes(row));
> > > >               htable.commit(batchupdate, rowLock);
> > > >               htable.unlockRow(rowLock);
> > > >
> > > > where the batchupdate only does a single "put" on a single column.
> > > >
> > > > The problem is that it is running very very slow when I use only 20
> > > > threads:
> > > > about 30 minutes against the 6~10 seconds of running with 10 threads.
> > > Also,
> > > > when using 20 threads, some *java.io.IOException:
> java.io.IOException:
> > > > Invalid row lock* messages are printed.
> > > >
> > > > Since it is the first time I'm using RowLocks, I'm not sure if the
> > > problem
> > > > is in my test or in HBase. So, I'm asking you for help.
> > > >
> > > > I'm using hbase 0.19.2 (rc) in pseudo-distributed mode on a mac book,
> > but
> > > > the behavior is the same when using 0.19.0. Logs, hbase-site.xml
> (which
> > > is
> > > > the default) and the test can be found on the following link:
> > > >
> > > > http://germoglio.googlepages.com/logs.zip
> > > >
> > > > Please notice that although this test doesn't make much sense of
> > locking,
> > > > the idea is to evolve it in order to perform a few operations besides
> > the
> > > > only single put between lockRow and unlockRow methods.
> > > >
> > > > Thank you very much,
> > > >
> > > > --
> > > > Guilherme
> > > >
> > > > msn: guigermoglio@hotmail.com
> > > > homepage: http://germoglio.googlepages.com
> > > >
> > >
> >
> >
> >
> > --
> > Guilherme
> >
> > msn: guigermoglio@hotmail.com
> > homepage: http://germoglio.googlepages.com
> >
>



-- 
Guilherme

msn: guigermoglio@hotmail.com
homepage: http://germoglio.googlepages.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message