hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: [jira] Commented: (HBASE-1500) KeyValue$KeyComparator array overrun
Date Fri, 12 Jun 2009 23:48:08 GMT
I don't want to pollute 1500 but I also noticed while running latest trunk that, when my test
table split, it split approximately like this:

    "" -> "70ee52bb9effbb6b5f085643602d95b"
    "70ee52bb9effbb6b5f085643602d95b" -> "73564360ebb6b5f08e52bb9eff2d95b"
    "73564360ebb6b5f08e52bb9eff2d95b" -> "76f2d95ebb64360b535f08e52bb9efb"
    "76f2d95ebb64360b535f08e52bb9efb" -> ""

even though the value sizes were equiprobable in the range [1,65535] and the keys were evenly
distributed in the keyspace, making the above split very likely incorrect (pointing to some
problem with finding the midkeys?). I don't know what to make of that, but if it is a datapoint
that will help someone, here it is...

   - Andy

From: Andrew Purtell <apurtell@apache.org>
To: hbase-dev@hadoop.apache.org
Sent: Friday, June 12, 2009 4:35:31 PM
Subject: Re: [jira] Commented: (HBASE-1500) KeyValue$KeyComparator array  overrun

If I roll one revision back from http://svn.apache.org/viewvc?view=rev&revision=782178
(via svn checkout -r782177) then I stop seeing the issues reported in HBASE-1500.

   - Andy

From: Ryan Rawson <ryanobjc@gmail.com>
To: hbase-dev@hadoop.apache.org
Sent: Friday, June 12, 2009 12:30:50 PM
Subject: Re: [jira] Commented: (HBASE-1500) KeyValue$KeyComparator array  overrun

More than 1 region suggests this is related to 1495....

On Jun 12, 2009 12:23 PM, "Andrew Purtell (JIRA)" <jira@apache.org> wrote:


Andrew Purtell commented on HBASE-1500:
Some detail I added up on IRC:

<apurtell>      i was wondering if it was just me...
<apurtell>      yeah would be great if someone else can confirm the repro
works for them...
<apurtell>      what's odd is the shell's count of all the rows comes back
before it even tries to advance the scanner to the next split. some kind of
META sort problem, parent region coming back before daughter? i'd like to
dig in but this code is all new to me
<apurtell>      in the repro case, the split is triggered and confirmed, so
there is more than one region...
<apurtell>      this is the same basic problem as with TableSplit, iterating
over META is going off the rails, giving bad results, triggering the out of
bounds exception
<apurtell>      at least this is what it looks like to me
<apurtell>      started immediately after commit of 1304
<apurtell>      noticed it first while testing stargate scanners, scanners
hang/crash/fail on tables with more than one region

> KeyValue$KeyComparator array overrun >
------------------------------------ > > ...
>            Assignee: ryan rawson

> Priority: Blocker > Fix For: 0.20.0 > > Attachments: hbase.log.gz, ...

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message