hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16890) Analyze the performance of AsyncWAL and fix the same
Date Wed, 26 Oct 2016 13:08:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15608401#comment-15608401
] 

ramkrishna.s.vasudevan commented on HBASE-16890:
------------------------------------------------

Some update from my side. I tried the disruptor way of handing over the Payload to the waitingConsumePayloads
instead of adding it to the waitingConsumePayloads in append/sync method. Am able to get 11%
increase in throughput with AsyncWAL with WALPE tool.
bq./hbase org.apache.hadoop.hbase.wal.WALPerformanceEvaluation -threads 100 -iterations 25000
-qualifiers 25 -keySize 50 -valueSize 200 

So with classic FSHLog it takes ~26000 ops/sec and with AsyncWAL it takes ~29500 to 30000
ops/sec.
The impl is simple but we have one problem which I solved in a hacky way.
In case of FSHLog we do generate a sequence number from the ring buffer for both append and
sync. And when we have a sync RingBufferTruck (RBT) we do take that as the currentSequence
number and update the highestSyncedTxid to that sequence and the remaining syncfutures are
checked if they are lesser than this currentSequence and only then mark it is done. So here
the updation of highestSyncedTxid  is actually done by the sync call.

But in case of AsyncWAL we have a case that the highestSyncedTxid  is also updated only after
we manually call a sync and also that sync is called on the last appended txid.  What ever
syncFuture was added for the sync call that is just to iterate and decide whether any of the
sync txid is less than the last appened txid and if so mark it as done. Here we are sure that
the sequenceId for the sync call is going to be one of them that was used for append.
For the RingBuff type that is not possible if for sync also we just  create a sequence id.
So the above logic will not mark the last sync call as done because the there is no append
call matching this sync's sequence id.

I did a hack (since I was running with PE only). It was nothing but if the syncFuture had
any sync call with txid that was one greater than the last appended txid I just marked it
as done. It is a big hack but still works for our case so that the WALPE gets completed. Else
it hangs on the last sync call.

> Analyze the performance of AsyncWAL and fix the same
> ----------------------------------------------------
>
>                 Key: HBASE-16890
>                 URL: https://issues.apache.org/jira/browse/HBASE-16890
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>    Affects Versions: 2.0.0
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>
>         Attachments: Screen Shot 2016-10-25 at 7.34.47 PM.png, Screen Shot 2016-10-25
at 7.39.07 PM.png, Screen Shot 2016-10-25 at 7.39.48 PM.png, async.svg, classic.svg, contention.png,
contention_defaultWAL.png
>
>
> Tests reveal that AsyncWAL under load in single node cluster performs slower than the
Default WAL. This task is to analyze and see if we could fix it.
> See some discussions in the tail of JIRA HBASE-15536.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message