hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17085) AsyncFSWAL may issue unnecessary AsyncDFSOutput.sync
Date Mon, 14 Nov 2016 13:01:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15663855#comment-15663855
] 

Duo Zhang commented on HBASE-17085:
-----------------------------------

[~ramkrishna] Could you please confirm your code again? Do you use the latest master and then
apply the v1 patch? I've added a log in sync method and tried a WALPE run. The txid of the
last SyncFuture in syncFutures maybe the same between calls, but the currentHighestProcessedAppendTxid
always increases.

Thanks.
{code}
LOG.info("=================" + fileLengthAtLastSync + ":" + currentHighestProcessedAppendTxid
+ ":" + (syncFutures.isEmpty() ? -1 : syncFutures.last().getTxid()));
{code}
fileLength:currentHighestProcessedAppendTxid:syncFutures.last().getTxid()
{noformat}
2016-11-14 20:57:38,644 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================18954972:16712758:16712902
2016-11-14 20:57:38,644 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19025752:16712778:16712902
2016-11-14 20:57:38,645 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19096532:16712798:16712902
2016-11-14 20:57:38,645 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19167312:16712819:16712902
2016-11-14 20:57:38,645 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19238092:16712839:16712902
2016-11-14 20:57:38,645 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19308872:16712857:16712902
2016-11-14 20:57:38,646 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19379652:16712879:16712902
2016-11-14 20:57:38,646 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19450432:16712899:16712902
2016-11-14 20:57:38,646 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19457510:16712900:16712902
2016-11-14 20:57:38,646 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19528290:16712921:16713043
2016-11-14 20:57:38,647 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19599070:16712941:16713078
2016-11-14 20:57:38,647 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19669850:16712960:16713078
2016-11-14 20:57:38,647 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19740630:16712980:16713078
2016-11-14 20:57:38,647 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19811410:16713000:16713078
2016-11-14 20:57:38,648 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19882190:16713020:16713078
2016-11-14 20:57:38,648 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================19952970:16713040:16713078
2016-11-14 20:57:38,648 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================20023750:16713060:16713078
2016-11-14 20:57:38,648 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================20080374:16713077:16713078
2016-11-14 20:57:38,649 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================20151154:16713097:16713248
2016-11-14 20:57:38,649 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================20221934:16713117:16713258
2016-11-14 20:57:38,650 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================20292714:16713137:16713258
2016-11-14 20:57:38,650 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================20363494:16713157:16713258
2016-11-14 20:57:38,650 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================20434274:16713177:16713258
2016-11-14 20:57:38,650 INFO  [AsyncFSWAL-pool1-t1] wal.AsyncFSWAL: =================20505054:16713196:16713258
{noformat}

> AsyncFSWAL may issue unnecessary AsyncDFSOutput.sync
> ----------------------------------------------------
>
>                 Key: HBASE-17085
>                 URL: https://issues.apache.org/jira/browse/HBASE-17085
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>    Affects Versions: 2.0.0
>            Reporter: Duo Zhang
>            Assignee: Duo Zhang
>             Fix For: 2.0.0
>
>         Attachments: HBASE-17085-v1.patch, HBASE-17085.patch
>
>
> The problem is in appendAndSync method, we will issue an  AsyncDFSOutput.sync if syncFutures
is not empty. The SyncFutures in syncFutures can only be removed after an AsyncDFSOutput.sync
comes back, so before the AsyncDFSOutput.sync actually returns, we will always issue an  AsyncDFSOutput.sync
after an append even if there is no new sync request.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message