hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7329) remove flush-related records from WAL
Date Sat, 22 Dec 2012 00:55:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538601#comment-13538601

Sergey Shelukhin commented on HBASE-7329:

The race condition is actually the same as the one already fixed for writer.append - we get
writer w/o updateLock, so it  s possible that it will be invalid by the time we used it. I
added the same handling in the writer as already done in append (catch NPE).

bq. On the patch, Sergey, are you convinced we can't make use of these edits? You think the
right thing to do is remove them rather than try and make use of this entry at WAL log splitting
time? It is the sequenceid of the last file flushed so we should replay all edits with a sequenceid
that is great that this one. At WAL log splitting time, I'm not sure how we'd find the last
one of these edits in a WAL without first reading all WALs... and if reading the WAL anyways,
might as well replay its edits? Or is that true? We could save a bunch not deserializing edits
we will discard anyways?
bq. Or, we don't need this edit because recently a mechanism was added where the regionserver
tells the master what its most recently flushed edit was and master knows by other means what
is the last flushed edit so this special edit it WALs is now redundant?
Yeah, that's HBASE-6508.

bq. We need to synchronize rolling now? Only one thread hereabouts anyways?
Hmm... is this guaranteed?

bq. Is this because you want to narrow the lock...to the body of cleanOldLogs?
The lock was actually used for two things - to avoid rolling log during flush, which is no
longer needed, and to safely operate on the map with oldest seqnums.
I narrowed the lock to the latter.
The lock was also taken in close(), for whatever reason. I will add a barrier to block flushes
and rolls after close is called.

bq. You mean to leave this in the patch
bq. + LOG.info("TMP: rollWriter has nothing to do");
No, will remove that.

> remove flush-related records from WAL
> -------------------------------------
>                 Key: HBASE-7329
>                 URL: https://issues.apache.org/jira/browse/HBASE-7329
>             Project: HBase
>          Issue Type: Improvement
>          Components: wal
>    Affects Versions: 0.96.0
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HBASE-7329-v0.patch, HBASE-7329-v0-tmp.patch
> Comments from many people in HBASE-6466 and HBASE-6980 indicate that flush records in
WAL are not useful. If so, they should be removed.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message