accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <>
Subject [jira] [Created] (ACCUMULO-2949) Write explicit "close" markers for WALs
Date Tue, 24 Jun 2014 19:00:25 GMT
Josh Elser created ACCUMULO-2949:

             Summary: Write explicit "close" markers for WALs
                 Key: ACCUMULO-2949
             Project: Accumulo
          Issue Type: Improvement
          Components: logger, replication
            Reporter: Josh Elser
            Assignee: Josh Elser

To ensure that WALs are not left in a dangling "open" state WRT replication, the garbage collector
scans the tablets and constructs a view of WALs that are currently in use. It consults that
view to determine which WALs can move to a "closed" replication state.

This isn't entirely correct because a WAL can "come back" again after being removed from a
Tablet. Consider the following:

# Table has one tablet hosted on one tserver
# Tablet gets some mutations
# Tablet gets MinC
# Tablet removes WAL entry as part of MinC
# WAL is "closed" WRT replication
# Tablet receives more mutations, starts using the same WAL

There are a couple of ways that this could present itself, each of which would result in re-replication
of data we've potentially already sent once. On an active system, I don't think this is of
big concern, and we already don't guarantee a "once and only once" replication contract so
this isn't critical.

If we want to move to explicit "end" tracking (see ACCUMULO-2835), we will need this implemented.

This message was sent by Atlassian JIRA

View raw message