hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Keith Thomas <keith.tho...@gmail.com>
Subject Re: Row lock issues with Indexed Tables
Date Tue, 15 Sep 2009 19:35:58 GMT

Jira created and patch attached:

   https://issues.apache.org/jira/browse/HBASE-1840


Clint Morgan-3 wrote:
> 
> I'm not sure if people have been using row locks with ITHBase. I havent
> been.
> 
> So looks like you found an issue, and have a solution for it. Sweet.
> Create
> a JIRA for it and attach your patch.
> 
> cheers,
> -clint
> 
> On Mon, Sep 14, 2009 at 12:52 PM, Keith Thomas
> <keith.thomas@gmail.com>wrote:
> 
>>
>> Apologies for the long post but being an HBase newbie I'd like a sanity
>> check
>> from wiser, more experienced folks with regards to my findings, and how
>> I've
>> fixed the issue I think I've found, before going down the path of
>> submitting
>> a formal patch.
>>
>> To re-create the problem I was seeing in my app I added the following
>> code
>> to TestIndexTable,
>>
>> import org.apache.hadoop.hbase.client.RowLock;
>>
>>  private void assertRowUpdated(int updatedRow, int expectedRowValue)
>>      throws IndexNotFoundException, IOException {
>>    ResultScanner scanner = table.getIndexedScanner(INDEX_COL_A, null,
>> null,
>>        null, null, null);
>>    byte[] persistedRowValue = null;
>>    for (Result rowResult : scanner) {
>>      byte[] row = rowResult.getRow();
>>      byte[] value = rowResult.getValue(COL_A);
>>      if
>>
>> (Bytes.toString(row).equals(Bytes.toString(PerformanceEvaluation.format(updatedRow))))
>> {
>>        persistedRowValue = value;
>>        LOG.info("update found: row [" + Bytes.toString(row)
>>          + "] value [" + Bytes.toString(value) + "]");
>>      }
>>      else
>>        LOG.info("updated index scan : row [" + Bytes.toString(row)
>>          + "] value [" + Bytes.toString(value) + "]");
>>    }
>>    scanner.close();
>>
>>
>>
>> Assert.assertEquals(Bytes.toString(PerformanceEvaluation.format(expectedRowValue)),
>>                                    Bytes.toString(persistedRowValue));
>>  }
>>
>>  private void updateRow(int row, int newValue) throws IOException {
>>      Put update = new Put(PerformanceEvaluation.format(row));
>>      byte[] valueA = PerformanceEvaluation.format(newValue);
>>      update.add(FAMILY, QUAL_A, valueA);
>>      table.put(update);
>>      LOG.info("Updated row [" + Bytes.toString(update.getRow()) + "] val:
>> ["
>>          + Bytes.toString(valueA) + "]");
>>  }
>>
>>  private void updateLockedRow(int row, int newValue) throws IOException {
>>      RowLock lock = table.lockRow(PerformanceEvaluation.format(row));
>>      Put update = new Put(PerformanceEvaluation.format(row), lock);
>>      byte[] valueA = PerformanceEvaluation.format(newValue);
>>      update.add(FAMILY, QUAL_A, valueA);
>>      LOG.info("Updating row [" + Bytes.toString(update.getRow()) + "]
>> val:
>> ["
>>          + Bytes.toString(valueA) + "]");
>>      table.put(update);
>>      LOG.info("Updated row [" + Bytes.toString(update.getRow()) + "] val:
>> ["
>>          + Bytes.toString(valueA) + "]");
>>      table.unlockRow(lock);
>>  }
>>
>>  public void testRowUpdate() throws IOException {
>>    writeInitalRows();
>>    int row = NUM_ROWS - 2;
>>    int value = MAX_VAL + 111;
>>    updateRow(row, value);
>>    assertRowUpdated(row, value);
>>  }
>>
>>  public void testLockedRowUpdate() throws IOException {
>>    writeInitalRows();
>>    int row = NUM_ROWS - 2;
>>    int value = MAX_VAL + 111;
>>    updateLockedRow(row, value);
>>    assertRowUpdated(row, value);
>>  }
>>
>> As I expected from the behavior of my own app the test testRowUpdate()
>> was
>> successful but the test testLockedRowUpdate() had an error as follows,
>>
>>    [junit] java.io.IOException: java.io.IOException: Invalid row lock
>>    [junit]     at
>> org.apache.hadoop.hbase.regionserver.HRegion.getLock(HRegion.java:1640)
>>    [junit]     at
>> org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1244)
>>    [junit]     at
>>
>> org.apache.hadoop.hbase.regionserver.tableindexed.IndexedRegion.put(IndexedRegion.java:97)
>>    [junit]     at
>> org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1216)
>>    [junit]     at
>>
>> org.apache.hadoop.hbase.regionserver.HRegionServer.put(HRegionServer.java:1818)
>>    [junit]     at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown
>> Source)
>>    [junit]     at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>    [junit]     at java.lang.reflect.Method.invoke(Method.java:597)
>>    [junit]     at
>> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650)
>>    [junit]     at
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
>>
>> NOTE: Line numbers in stacktrace may not make sense because I've been
>> hacking in loads of debug info.
>>
>> To fix the error I made three changes to IndexedRegion.java. The changes
>> are
>> on lines marked *KST* below and in summary all they do is propagate the
>> lockid.
>>
>>  public void put(Put put, Integer lockId, boolean writeToWAL)
>>      throws IOException {
>>    updateIndexes(put, lockId); // Do this first because will want to see
>> the old row *KST*
>>    super.put(put, lockId, writeToWAL);
>>  }
>>
>>  private void updateIndexes(Put put, Integer lockId) throws IOException {
>> // *KST*
>>    List<IndexSpecification> indexesToUpdate = new
>> LinkedList<IndexSpecification>();
>>
>>    // Find the indexes we need to update
>>    for (IndexSpecification index : getIndexes()) {
>>      if (possiblyAppliesToIndex(index, put)) {
>>        indexesToUpdate.add(index);
>>      }
>>    }
>>
>>    if (indexesToUpdate.size() == 0) {
>>      return;
>>    }
>>
>>    NavigableSet<byte[]> neededColumns =
>> getColumnsForIndexes(indexesToUpdate);
>>    NavigableMap<byte[], byte[]> newColumnValues = getColumnsFromPut(put);
>>
>>    Get oldGet = new Get(put.getRow());
>>    for (byte [] neededCol : neededColumns) {
>>      oldGet.addColumn(neededCol);
>>    }
>>
>>    Result oldResult = super.get(oldGet, lockId); // *KST*
>>
>> Feedback as to how I should proceed with these findings would be great.
>> e.g.
>> should I submit a formal patch, have I made a silly coding error in my
>> test
>> or in IndexedRegion? Also, maybe the test are insufficient. I have al
>> sorts
>> of variations I have tried with autoflush turned off, maybe I should
>> include
>> those tests too?
>> --
>> View this message in context:
>> http://www.nabble.com/Row-lock-issues-with-Indexed-Tables-tp25442355p25442355.html
>> Sent from the HBase User mailing list archive at Nabble.com.
>>
>>
> 
> 

-- 
View this message in context: http://www.nabble.com/Row-lock-issues-with-Indexed-Tables-tp25442355p25460365.html
Sent from the HBase User mailing list archive at Nabble.com.


Mime
View raw message