hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Pearson (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-855) compaction can return less versions then we should in some cases
Date Sat, 30 Aug 2008 03:09:44 GMT

     [ https://issues.apache.org/jira/browse/HBASE-855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Billy Pearson updated HBASE-855:
--------------------------------

    Attachment: 855-patch.txt

This should fix a problem I seen in the max version and ttl

stack commented on HBASE-834 that he still seeing old version of data in .META. historian

I looked around and thank I might have found a second bug that might be the problem and included
it in the patch

When we see a new row/column we where using the var timesSeen to track the times we have seen
the same row/column.
but on a new row/column we set it to 0 on the first new row/column seen
then below we where check for max versions limit like this
if (timesSeen <= family.getMaxVersions()

With the timesSeen set to 0 that would allow 1 extra cell to pass on to the HStoreFile then
the max versions setting
We should have been setting timesSeen to 1 in place of 0 sense its the first row/column seen
not 0

I thank this might be your problem you are seeing stack when the record is 
deleted there could still be one old cell hidden in the table that will not show up until
the newest cell is deleted.


> compaction can return less versions then we should in some cases
> ----------------------------------------------------------------
>
>                 Key: HBASE-855
>                 URL: https://issues.apache.org/jira/browse/HBASE-855
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: regionserver
>    Affects Versions: 0.2.1
>            Reporter: Billy Pearson
>            Assignee: Billy Pearson
>             Fix For: 0.2.1, 0.18.0
>
>         Attachments: 855-patch.txt
>
>
> say we have a column with max version = 3 and we have 3 records  
> we insert a new record with a old timestamp.
> What happeds in the compaction is the the new record with the old timestamp get read
first and could push out some of our 
> versions if the new record(s) with the old timestamp has a expired ttl.
> This happens because we track the total times we see a row/column but do not reduce this
count if the cell is expired
> and sense we pass the cell in order of the newest HStoreFile first with the newest records
passed might not be the newest timestamps.
> Got to wait for HBASE-834 to be committed then I can add a patch for this bug. will be
a simple fix.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message