hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17085) AsyncFSWAL may issue unnecessary AsyncDFSOutput.sync
Date Mon, 14 Nov 2016 13:21:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663897#comment-15663897

ramkrishna.s.vasudevan commented on HBASE-17085:

I tried setting 128MB but still not consistent results. But may be need to check with bigger
data size. In your test can you change the batch size and run PE? If it performs better it
is good.
And in your patch there is a suggestion. Pls check
 if(!syncFutures.isEmpty()) {
      long lastTxid = syncFutures.last().getTxid();
      if (lastTxid > highestProcessedAppendTxidAtLastSync) {
        long temp = newHighestProcessedAppendTxid;
        if((temp + 1) > lastTxid)
        // we have at least one sync request
See if this is fine. The reason is that we can aggregate more before calling sync.

> AsyncFSWAL may issue unnecessary AsyncDFSOutput.sync
> ----------------------------------------------------
>                 Key: HBASE-17085
>                 URL: https://issues.apache.org/jira/browse/HBASE-17085
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>    Affects Versions: 2.0.0
>            Reporter: Duo Zhang
>            Assignee: Duo Zhang
>             Fix For: 2.0.0
>         Attachments: HBASE-17085-v1.patch, HBASE-17085.patch
> The problem is in appendAndSync method, we will issue an  AsyncDFSOutput.sync if syncFutures
is not empty. The SyncFutures in syncFutures can only be removed after an AsyncDFSOutput.sync
comes back, so before the AsyncDFSOutput.sync actually returns, we will always issue an  AsyncDFSOutput.sync
after an append even if there is no new sync request.

This message was sent by Atlassian JIRA

View raw message