accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (JIRA)" <>
Subject [jira] [Created] (ACCUMULO-3423) speed up WAL roll-overs
Date Mon, 15 Dec 2014 22:39:13 GMT
Eric Newton created ACCUMULO-3423:

             Summary: speed up WAL roll-overs
                 Key: ACCUMULO-3423
             Project: Accumulo
          Issue Type: Bug
          Components: master, tserver
            Reporter: Eric Newton

After reading the proposal on HBASE-10278, I realized there are many ways to make the Accumulo
WAL roll-over faster.

# Open two WALogs, but use only one until it reaches the WALog roll-over size
# Rollover consists only of swapping the writers
# WALog roll consists of the final close, which can happen in parallel
# Don't mark the tablets with log entries: they are already marked with the tserver
# The tserver can make notes about the logs-in-use in the metadata table(s) as part of opening
the log.
# The master can copy the log entries to tablets while unassigning them,  piggybacking on
the unassigment mutation.
# Tablet servers can remove their current log entries from the metadata tables when they have
no tablets using them.

There are two issues: 
# tablets will have an empty file in recovery, nearly all the time, but the recovery code
already handles that case.  
# presently, a tablet doesn't have a marker for a log it did not use. Many more tablets will
attempt to recover when it is unnecessary.

This would also address ACCUMULO-2889.

This message was sent by Atlassian JIRA

View raw message