hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HBASE-1761) getclosest doesn't understand delete family; manifests as "HRegionInfo was null or empty in .META" A.K.A the BS problem
Date Mon, 17 Aug 2009 17:52:14 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack resolved HBASE-1761.
--------------------------

      Resolution: Fixed
    Hadoop Flags: [Reviewed]

Committed trunk and branch.  Here's commit message:

{code}
Rewritten getclosestatorbefore that works with new style deletes (delete
family, delete column and just delete.  Doesn't carry as much load as it
goes from memstore across storefiles (e.g. only one candidate is
carried rather than all as previous) and always goes for the target row
rather than zig-zag in so should perform a bit better.  Includes fix
that should make it so we no longer return rows from wrong table.  Also
add flush on drop of table.

M src/test/org/apache/hadoop/hbase/HBaseTestCase.java
  Formatting.
A src/test/org/apache/hadoop/hbase/regionserver/TestGetClosestAtOrBefore.java
  Added testing of getClosestAtOrBefore.  Most of this test was in hbase before
  hbase-1304 went in.  Also dropped what was in TestMemStore testing 
  getClosest and moved it here instead.
M src/test/org/apache/hadoop/hbase/regionserver/TestMemStore.java
  Made a little more stringent asserting row count same as column count.
  Added new test to assert we can see new values as they come in as we scan
  (More hbase-1738).
  MemStore constructor no longer takes ttl.  Removed the testGetClosest from
  here and moved up into TestGetClosestAtOrBefore.
M src/test/org/apache/hadoop/hbase/regionserver/TestHRegion.java
  Remove unused imports.
M src/test/org/apache/hadoop/hbase/regionserver/TestCompaction.java
  Formatting.
A src/java/org/apache/hadoop/hbase/regionserver/GetClosestRowBeforeTracker.java
  New class that holds getclosest state and utility.  Like ScanDeleteTracker
  and GetDeleteTracker but doens't implement the DeleteTracker interface
  as it needs to span rows.
M src/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
  Don't allocate ArrayList unless we're going to use it.
M  src/java/org/apache/hadoop/hbase/regionserver/GetDeleteTracker.java
  Fix the isEmpty.
M src/java/org/apache/hadoop/hbase/regionserver/MemStore.java
  No longer takes a ttl in constructor (It now lives in GetClosestRowBeforeTracker).
  Reimplemented getClosestRowAtOrBefore.
M  src/java/org/apache/hadoop/hbase/regionserver/HRegion.java
  Reimplemented getClosestRowAtOrBefore.
M  src/java/org/apache/hadoop/hbase/regionserver/Store.java
  MemStore doesn't take a ttl.  Reimplemented getClosestRowAtOrBefore.
M  src/java/org/apache/hadoop/hbase/KeyValue.java
  Added comparator that only looks at row portion of a KV.  Useful for
  case where want to use kv rather than extract its row as row key in map.
  Changed the type on KVs that do not have values to be Type.Maximum by default.
  (Was Type.Put by default).  Removed cloneRow.  Not a clone as Ryan pointed
  out and can use constructor instead.
  (isDelete, isDeleteFamily): Added.
M src/java/org/apache/hadoop/hbase/util/Bytes.java
  Formatting.
M src/java/org/apache/hadoop/hbase/client/HConnectionManager.java
  Log message.
M src/java/org/apache/hadoop/hbase/client/Put.java
  Was allocating a Set twice on each Put deserialization.
M  bin/HBase.rb
  Flush before major compacting when dropping table.
{code}

> getclosest doesn't understand delete family; manifests as "HRegionInfo was null or empty
in .META" A.K.A the BS problem
> -----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-1761
>                 URL: https://issues.apache.org/jira/browse/HBASE-1761
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>             Fix For: 0.20.0
>
>         Attachments: 1761-v10.patch, 1761-v11.patch, 1761-v12.patch, 1761-v13.patch,
1761-v2.patch, 1761-v3.patch, 1761-v4.patch, 1761-v5.patch, 1761-v6.patch, 1761-v7.patch,
1761-v9.patch, 1761.patch
>
>
> getclosestatorbefore was not converted to deal with the new delete types.  It only knows
how to process old style deletes.  Usually all is well as edits come in but its possible to
get into state where you have persisted in one file a deletefamily for all in meta and in
the file behind it, there are entries on the info family.  Since closest doesn't understand
deletefamily, it will return the Put rows only for the subsequent getfull, which knows how
to work with deletefamilies fail.
> Once this happens, table is hosed.  Seen on Bradford Stephens upload and at Powerset.
 "Fix" is flush and major compact.  Gives impression that hbase is 'delicate'.  Fixing.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message