hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-1738) Scanner doesnt reset when a snapshot is created, could miss new updates into the 'kvset' (active part)
Date Mon, 17 Aug 2009 20:16:14 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-1738:
-------------------------

    Attachment: 1738-v3.patch

Here is v3:

{code}
M src/test/org/apache/hadoop/hbase/regionserver/TestMemStore.java
  Reset memstore scanners at end of each full scan.
  Reenable part of test that was broken because we weren't getting
  all values.
M src/java/org/apache/hadoop/hbase/regionserver/MemStore.java
  Have MemStoreScanners become Observers on MemStore.  If MemStore
  snapshots or clears snapshots, have Scanners update themselves.
  Changes MemStore to have a single scanner for both kvset and for
  snapshot rather than one for each.  Now we meld the results from
  both in the one scanner.
  New inner FirstOnRow class that is updated atomically.
{code}

> Scanner doesnt reset when a snapshot is created, could miss new updates into the 'kvset'
(active part)
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-1738
>                 URL: https://issues.apache.org/jira/browse/HBASE-1738
>             Project: Hadoop HBase
>          Issue Type: Bug
>    Affects Versions: 0.20.0
>            Reporter: ryan rawson
>            Assignee: stack
>             Fix For: 0.20.0
>
>         Attachments: 1738-v2.patch, 1738-v3.patch, 1738.patch
>
>
> when a Scanner is created, it creates 2 MemStoreScanners on the kvset and the snapshot
(internal names of Memstore)... if the snapshot is originally empty, it only creates the 1,
for kvset.  When the snapshot is created, the outstanding Scanners now have a pointer to the
tree that is now the snapshot, but no pointer to the kvset.
> When the flush completes, the scanner will reset the memstore scanners and 'see' the
new values again.
> If there is a large delay between snapshot and finalization of the flush, there can be
a large period of time a scanner doesnt see 'new' values that are being inserted. the canonical
'bad' case where this can do things is the META scanner, and we end up with double assignment.
> The snapshot is really lightweight, it only takes out a small lock in memstore, so im
not sure there is an easy mechanism to hook to without building out a bit more code or restructuring
the memstore scanner.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message