hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amitanand Aiyer (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4485) Eliminate window of missing Data
Date Thu, 20 Oct 2011 18:00:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131844#comment-13131844

Amitanand Aiyer commented on HBASE-4485:

I am not clear on weather we need the notifyChangedReader to be under the lock or not? I guess
that depends on the
state that the lock is trying to guard. My understanding/assumption was that 

(a) The lock in Store.java  is guarding the list-of-store-files plus the state of MemStore's
(kvset and snapshot) references. 

(b) Was also assuming that StoreScanner is okay with seeing an older older set of files, as
long as the memStoreScanner that
it is using is current (i.e. The StoreScanner's view of the world may not be accurate as of
"now". But it is guaranteed
to be consistent across the set of storefiles and the memstore (at the time getScanners was
called to create the scanners).

I don't think that we can totally avoid (b) even if we have notifyChangedReaders under the
lock. StoreScanner could already
be processing a next() operation, during which the updateReader will just have to wait to
let the StoreScanner complete, however
long it takes.

Please correct me if my assumptions/understanding are wrong.

> Eliminate window of missing Data
> --------------------------------
>                 Key: HBASE-4485
>                 URL: https://issues.apache.org/jira/browse/HBASE-4485
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Amitanand Aiyer
>            Assignee: Amitanand Aiyer
>             Fix For: 0.94.0
>         Attachments: 4485-v1.diff, 4485-v2.diff, 4485-v3.diff, 4485-v4.diff, repro_bug-4485.diff
> After incorporating v11 of the 2856 fix, we discovered that we are still having some
ACID violations.
> This time, however, the problem is not about including "newer" updates; but, about missing
older updates
> that should be including. 
> Here is what seems to be happening.
> There is a race condition in the StoreScanner.getScanners()
>   private List<KeyValueScanner> getScanners(Scan scan,
>       final NavigableSet<byte[]> columns) throws IOException {
>     // First the store file scanners
>     List<StoreFileScanner> sfScanners = StoreFileScanner
>       .getScannersForStoreFiles(store.getStorefiles(), cacheBlocks,
>                                 isGet, false);
>     List<KeyValueScanner> scanners =
>       new ArrayList<KeyValueScanner>(sfScanners.size()+1);
>     // include only those scan files which pass all filters
>     for (StoreFileScanner sfs : sfScanners) {
>       if (sfs.shouldSeek(scan, columns)) {
>         scanners.add(sfs);
>       }
>     }
>     // Then the memstore scanners
>     if (this.store.memstore.shouldSeek(scan)) {
>       scanners.addAll(this.store.memstore.getScanners());
>     }
>     return scanners;
>   }
> If for example there is a call to Store.updateStorefiles() that happens between
> the store.getStorefiles() and this.store.memstore.getScanners(); then
> it is possible that there was a new HFile created, that is not seen by the
> StoreScanner, and the data is not present in the Memstore.snapshot either.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message