hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-621) Make MAX_VERSIONS work like TTL: In scans and gets, check MAX_VERSIONs setting and return that many only rather than wait on compaction
Date Tue, 20 May 2008 16:17:55 GMT

    [ https://issues.apache.org/jira/browse/HBASE-621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12598346#action_12598346
] 

Jean-Daniel Cryans commented on HBASE-621:
------------------------------------------

I have been digging around in HStore and here is what I found that causes me problem or that
I think should get someone's attention :

- stack mentionned that this code around line 1250 will maybe need to change : 

{code}
if (results.size() == numVersions) {
        return results.toArray(new Cell[results.size()]);
}
{code}

I thought the same but by looking in Memcache we can see that the TTL problem is already handled
:

{code}
if (ttl == HConstants.FOREVER ||
         now < itKey.getTimestamp() + ttl) {
     result.add(new Cell(tailMap.get(itKey), itKey.getTimestamp()));
 } else {
      victims.add(itKey);
...
// Remove expired victims from the map.
for (HStoreKey v: victims)
  map.remove(v);
{code}

So I guess that it's ok to leave it like that. 

- I was wondering where to put tests for this modification and I found that regionserver.TestCompaction
tests conflicts with my modification. Have a look at :

{code}
    // Add more content.  Now there are about 5 versions of each column.
    // Default is that there only 3 (MAXVERSIONS) versions allowed per column.
    // Assert > 3 and then after compaction, assert that only 3 versions
    // available.
    addContent(new HRegionIncommon(r), Bytes.toString(COLUMN_FAMILY));
    Cell[] cellValues = 
      r.get(STARTROW, COLUMN_FAMILY_TEXT, 100 /*Too many*/);
    // Assert that I can get > 5 versions (Should be at least 5 in there).
    assertTrue(cellValues.length >= 5);
{code}

This now fails because it returns MAXVERSIONS versions. Any advices on how to handle the max
version problem? Is there an easy way of changing it just for this test?

Also, is TestCompaction a good place to test this jira? (since it seems it is already handled)

> Make MAX_VERSIONS work like TTL: In scans and gets, check MAX_VERSIONs setting and return
that many only rather than wait on compaction
> ---------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-621
>                 URL: https://issues.apache.org/jira/browse/HBASE-621
>             Project: Hadoop HBase
>          Issue Type: Improvement
>            Reporter: stack
>            Assignee: Jean-Daniel Cryans
>             Fix For: 0.2.0
>
>
> HBASE-47 added specification of TTL on cells.  The implementation checks cell timestamp
against configured TTL before returning results scanning or getting.  You can also set the
maximum versions of a cell to keep.  The maximum versions is not checked scanning or getting,
only when we compact (We'll drop cells that are beyond the maximum version at compaction time).
 This issue is about adding check for maximum versions to gets and scans so that if you ask
for all versions but have configured the store to only keep 3 versions, though 4 may have
been inserted, you'll currently get 4 returned (if compactions have not had a chance to run).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message