hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10519) Add a configuration option to enable in-progress edit log tailing
Date Wed, 20 Jul 2016 23:34:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386804#comment-15386804

Andrew Wang commented on HDFS-10519:

Thanks for revving Jiayi. Overall looks good, mostly just nits now. I think the next rev should
do it.

* in DFSConfigKeys, let's group the new config key with the other "dfs.ha.tail-edits" keys
* Regarding {{isBoundedByDurableTxId}}, maybe shorten to {{onlyDurableTxns}}? Should update
javadoc for JournalSet#selectInputStreams also to talk about the boolean. Would be good to
explain what a "durable TxId" means also, and the tightness of the bound (it can be conservative).
* Nit: in QuorumJournalManager, we already assigned {{remoteLog.getEditTxId()}} to endTxId.
So we can re-use {{endTxId}} when doing Math.min.
* QuorumOutputStream, recommend we add the new boolean at the end of the list of parameters,
we generally try to put flags/options at the end
* RemoteEditLogManifest, the manifest might not have any logs, then the {{logs.get(0)}} check
will fail.

I would like to think about those randomized tests more, but yea that can happen in another

> Add a configuration option to enable in-progress edit log tailing
> -----------------------------------------------------------------
>                 Key: HDFS-10519
>                 URL: https://issues.apache.org/jira/browse/HDFS-10519
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: ha
>            Reporter: Jiayi Zhou
>            Assignee: Jiayi Zhou
>            Priority: Minor
>         Attachments: HDFS-10519.001.patch, HDFS-10519.002.patch, HDFS-10519.003.patch,
HDFS-10519.004.patch, HDFS-10519.005.patch, HDFS-10519.006.patch, HDFS-10519.007.patch, HDFS-10519.008.patch
> Standby Namenode has the option to do in-progress edit log tailing to improve the data
freshness. In-progress tailing is already implemented, but it's not enabled as default configuration.
And there's no related configuration key to turn it on.
> Adding a related configuration key to let Standby Namenode is reasonable and would be
a basis for further improvement on Standby Namenode.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message