hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-792) Rewrite getClosestAtOrJustBefore; doesn't scale as currently written
Date Wed, 21 Jan 2009 23:00:09 GMT

     [ https://issues.apache.org/jira/browse/HBASE-792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

stack updated HBASE-792:

         Priority: Blocker  (was: Major)
    Fix Version/s: 0.20.0

This can actually be responsible for slowing down whole cluster (J-D saw it in 0.18 hbase
up on his openspaces cluster)

> Rewrite getClosestAtOrJustBefore; doesn't scale as currently written
> --------------------------------------------------------------------
>                 Key: HBASE-792
>                 URL: https://issues.apache.org/jira/browse/HBASE-792
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>            Priority: Blocker
>             Fix For: 0.20.0
>         Attachments: 792.patch
> As currently written, as a table gets bigger, the number of rows .META. needs to keep
count of grows.
> As written, our getClosestAtOrJustBefore, goes through every storefile and in each picks
up any row that could be a possible candidate for closest before.  It doesn't just get the
closest from the storefile, but all keys that are closest before.  Its not selective because
how can it tell at the store file level which of the candidates will survive deletes that
are sitting in later store files or up in memcache.
> So, if a store file has keys 0-10 and we ask to get the row that is closest or just before
7, it returns rows 0-7.. and so on per store file.
> Can bet big and slow weeding key wanted.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message