hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16890) Analyze the performance of AsyncWAL and fix the same
Date Tue, 25 Oct 2016 05:17:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604232#comment-15604232

Duo Zhang commented on HBASE-16890:

There is no contention for the synchronization in DFSClient#DFSOS for our FSHLog. We always
call it in the consumer thread of Disruptor...

The payload queue in AsyncFSWAL acts as the ringbuffer in FSHLog.

And I still do not think calculate checksum when syncing cause any difference. For both FSHLog
and AsyncFSWAL, the append and sync are executed in one thread, no concurrency at all. The
FSHLog.append only puts the WALEntry into the ringbuffer, not appends it to DFSClient#DFSOS.
It does not make sense that calcuate the checksum of a big piece of data chunk by chunk could
be faster? The calculated amount does not change I think?


> Analyze the performance of AsyncWAL and fix the same
> ----------------------------------------------------
>                 Key: HBASE-16890
>                 URL: https://issues.apache.org/jira/browse/HBASE-16890
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>         Attachments: contention.png, contention_defaultWAL.png
> Tests reveal that AsyncWAL under load in single node cluster performs slower than the
Default WAL. This task is to analyze and see if we could fix it.
> See some discussions in the tail of JIRA HBASE-15536.

This message was sent by Atlassian JIRA

View raw message