hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clint Morgan <clint.mor...@troove.net>
Subject Re: Row lock issues with Indexed Tables
Date Tue, 15 Sep 2009 01:31:53 GMT
I'm not sure if people have been using row locks with ITHBase. I havent
been.

So looks like you found an issue, and have a solution for it. Sweet. Create
a JIRA for it and attach your patch.

cheers,
-clint

On Mon, Sep 14, 2009 at 12:52 PM, Keith Thomas <keith.thomas@gmail.com>wrote:

>
> Apologies for the long post but being an HBase newbie I'd like a sanity
> check
> from wiser, more experienced folks with regards to my findings, and how
> I've
> fixed the issue I think I've found, before going down the path of
> submitting
> a formal patch.
>
> To re-create the problem I was seeing in my app I added the following code
> to TestIndexTable,
>
> import org.apache.hadoop.hbase.client.RowLock;
>
>  private void assertRowUpdated(int updatedRow, int expectedRowValue)
>      throws IndexNotFoundException, IOException {
>    ResultScanner scanner = table.getIndexedScanner(INDEX_COL_A, null, null,
>        null, null, null);
>    byte[] persistedRowValue = null;
>    for (Result rowResult : scanner) {
>      byte[] row = rowResult.getRow();
>      byte[] value = rowResult.getValue(COL_A);
>      if
>
> (Bytes.toString(row).equals(Bytes.toString(PerformanceEvaluation.format(updatedRow))))
> {
>        persistedRowValue = value;
>        LOG.info("update found: row [" + Bytes.toString(row)
>          + "] value [" + Bytes.toString(value) + "]");
>      }
>      else
>        LOG.info("updated index scan : row [" + Bytes.toString(row)
>          + "] value [" + Bytes.toString(value) + "]");
>    }
>    scanner.close();
>
>
>
> Assert.assertEquals(Bytes.toString(PerformanceEvaluation.format(expectedRowValue)),
>                                    Bytes.toString(persistedRowValue));
>  }
>
>  private void updateRow(int row, int newValue) throws IOException {
>      Put update = new Put(PerformanceEvaluation.format(row));
>      byte[] valueA = PerformanceEvaluation.format(newValue);
>      update.add(FAMILY, QUAL_A, valueA);
>      table.put(update);
>      LOG.info("Updated row [" + Bytes.toString(update.getRow()) + "] val:
> ["
>          + Bytes.toString(valueA) + "]");
>  }
>
>  private void updateLockedRow(int row, int newValue) throws IOException {
>      RowLock lock = table.lockRow(PerformanceEvaluation.format(row));
>      Put update = new Put(PerformanceEvaluation.format(row), lock);
>      byte[] valueA = PerformanceEvaluation.format(newValue);
>      update.add(FAMILY, QUAL_A, valueA);
>      LOG.info("Updating row [" + Bytes.toString(update.getRow()) + "] val:
> ["
>          + Bytes.toString(valueA) + "]");
>      table.put(update);
>      LOG.info("Updated row [" + Bytes.toString(update.getRow()) + "] val:
> ["
>          + Bytes.toString(valueA) + "]");
>      table.unlockRow(lock);
>  }
>
>  public void testRowUpdate() throws IOException {
>    writeInitalRows();
>    int row = NUM_ROWS - 2;
>    int value = MAX_VAL + 111;
>    updateRow(row, value);
>    assertRowUpdated(row, value);
>  }
>
>  public void testLockedRowUpdate() throws IOException {
>    writeInitalRows();
>    int row = NUM_ROWS - 2;
>    int value = MAX_VAL + 111;
>    updateLockedRow(row, value);
>    assertRowUpdated(row, value);
>  }
>
> As I expected from the behavior of my own app the test testRowUpdate() was
> successful but the test testLockedRowUpdate() had an error as follows,
>
>    [junit] java.io.IOException: java.io.IOException: Invalid row lock
>    [junit]     at
> org.apache.hadoop.hbase.regionserver.HRegion.getLock(HRegion.java:1640)
>    [junit]     at
> org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1244)
>    [junit]     at
>
> org.apache.hadoop.hbase.regionserver.tableindexed.IndexedRegion.put(IndexedRegion.java:97)
>    [junit]     at
> org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1216)
>    [junit]     at
>
> org.apache.hadoop.hbase.regionserver.HRegionServer.put(HRegionServer.java:1818)
>    [junit]     at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown
> Source)
>    [junit]     at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>    [junit]     at java.lang.reflect.Method.invoke(Method.java:597)
>    [junit]     at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650)
>    [junit]     at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
>
> NOTE: Line numbers in stacktrace may not make sense because I've been
> hacking in loads of debug info.
>
> To fix the error I made three changes to IndexedRegion.java. The changes
> are
> on lines marked *KST* below and in summary all they do is propagate the
> lockid.
>
>  public void put(Put put, Integer lockId, boolean writeToWAL)
>      throws IOException {
>    updateIndexes(put, lockId); // Do this first because will want to see
> the old row *KST*
>    super.put(put, lockId, writeToWAL);
>  }
>
>  private void updateIndexes(Put put, Integer lockId) throws IOException {
> // *KST*
>    List<IndexSpecification> indexesToUpdate = new
> LinkedList<IndexSpecification>();
>
>    // Find the indexes we need to update
>    for (IndexSpecification index : getIndexes()) {
>      if (possiblyAppliesToIndex(index, put)) {
>        indexesToUpdate.add(index);
>      }
>    }
>
>    if (indexesToUpdate.size() == 0) {
>      return;
>    }
>
>    NavigableSet<byte[]> neededColumns =
> getColumnsForIndexes(indexesToUpdate);
>    NavigableMap<byte[], byte[]> newColumnValues = getColumnsFromPut(put);
>
>    Get oldGet = new Get(put.getRow());
>    for (byte [] neededCol : neededColumns) {
>      oldGet.addColumn(neededCol);
>    }
>
>    Result oldResult = super.get(oldGet, lockId); // *KST*
>
> Feedback as to how I should proceed with these findings would be great.
> e.g.
> should I submit a formal patch, have I made a silly coding error in my test
> or in IndexedRegion? Also, maybe the test are insufficient. I have al sorts
> of variations I have tried with autoflush turned off, maybe I should
> include
> those tests too?
> --
> View this message in context:
> http://www.nabble.com/Row-lock-issues-with-Indexed-Tables-tp25442355p25442355.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message