phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vincent Poon (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-4530) Do not collect delete markers during major compaction of table with disabled mutable indexes
Date Wed, 21 Feb 2018 18:46:00 GMT


Vincent Poon commented on PHOENIX-4530:

[~jamestaylor] In the code, preCompactScannerOpen() is called before preCompact().  Both
return a scanner.  As long as preCompact() does not replace the scanner entirely with a new
one, then it'll still have the settings from preCompactScannerOpen().  Our code just wraps
the scanner with a delegate pattern, so it should be fine.

Just noticed with HBase 2.0+, things have changed where preCompactScannerOpen does not return
a scanner - only preCompact does, so there's less ambiguity there.  However, the javadoc
for preCompactScannerOpen() does indicate that changes to versions/TTL should be made there,
so I think it's correct to put our logic in preCompactScannerOpen().

> Do not collect delete markers during major compaction of table with disabled mutable
> --------------------------------------------------------------------------------------------
>                 Key: PHOENIX-4530
>                 URL:
>             Project: Phoenix
>          Issue Type: Improvement
>         Environment:  
>            Reporter: James Taylor
>            Assignee: Vincent Poon
>            Priority: Major
>         Attachments: PHOENIX-4530.master.v1.patch, PHOENIX-4530.master.v2.patch
> If major compaction occurs on a table with mutable indexes that have the INDEX_DISABLE_TIMESTAMP
set, we currently permanently disable the index, forcing it to be manually rebuilt from scratch.
This is to prevent it from potentially being corrupted as we need the delete markers to remain
in order to guarantee the data table and index table remain in sync.
> An alternate approach (mentioned by [] during review) is to detect this
case in a pre-compaction hook and set the compaction up so that delete markers are not removed.
This would have the advantage that we wouldn't have to permanently disable the index and rebuild
it from scratch.

This message was sent by Atlassian JIRA

View raw message