hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-7329) remove flush-related records from WAL and reduce locking
Date Wed, 16 Jan 2013 02:13:13 GMT

     [ https://issues.apache.org/jira/browse/HBASE-7329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sergey Shelukhin updated HBASE-7329:
------------------------------------

    Attachment: HBASE-7329-v2.patch

This is the detailed description of what this patch does.

First, it removes memstore flush WAL record (as advertised).

Second, it deals with locking. Previously, there existed cacheFlushLock, which was used for
4 distinct purposes:
1) Make sure the WAL record for memstore flush ends up in the same WAL that was active when
the flush started. In other words, prevent log rolling during memstore flush.
2) Make sure only one log rolling operation happens at a time.
3) Control access to "lastSeqWritten" map for any operation that is not covered by SynchronizedMap.
4) Make sure that close() waits for all the flushes and rolls, by virtue of doing bunch of
stuff under this lock and setting .closed = true for potential future callers.
In fact, I am not sure what prevented startCacheFlush from proceeding after close is done,
since it doesn't perform the check of closed.

This was replaced by more granular/appropriate locking:
1) No longer needed/relevant, in fact there's HBASE-7011 to get rid of this too.
2) rollWriter was made "synchronized".
3) Control of the map is under its own small-scope lock.
4) Added a simple barrier class to be able to block close on outstanding operations, and drain
existing ones.

Third, some cleanup.

I am (re)running medium+ tests now.
                
> remove flush-related records from WAL and reduce locking
> --------------------------------------------------------
>
>                 Key: HBASE-7329
>                 URL: https://issues.apache.org/jira/browse/HBASE-7329
>             Project: HBase
>          Issue Type: Improvement
>          Components: wal
>    Affects Versions: 0.96.0
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HBASE-7329-v0.patch, HBASE-7329-v0.patch, HBASE-7329-v0-tmp.patch,
HBASE-7329-v1.patch, HBASE-7329-v1.patch, HBASE-7329-v2.patch
>
>
> Comments from many people in HBASE-6466 and HBASE-6980 indicate that flush records in
WAL are not useful. If so, they should be removed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message