hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13189) Standby NameNode should roll active edit log when checkpointing
Date Mon, 26 Feb 2018 14:25:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376939#comment-16376939
] 

Kihwal Lee commented on HDFS-13189:
-----------------------------------

We do configure auto rolling as you mentioned. A standby NN can be down during a maintenance
or for any reasons, so this seems to be a safer option than rely solely on the standby.

> Standby NameNode should roll active edit log when checkpointing
> ---------------------------------------------------------------
>
>                 Key: HDFS-13189
>                 URL: https://issues.apache.org/jira/browse/HDFS-13189
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: Chao Sun
>            Priority: Minor
>
> When the SBN is doing checkpointing, it will hold the {{cpLock}}. In the current implementation
of edit log tailer thread, it will first check and roll active edit log, and then tail and
apply edits. In the case of checkpointing, it will be blocked on the {{cpLock}} and will not
roll the edit log.
> It seems there is no dependency between the edit log roll and tailing edits, so a better
may be to do these in separate threads. This will be helpful for people who uses the observer
feature without in-progress edit log tailing. 
> An alternative is to configure {{dfs.namenode.edit.log.autoroll.multiplier.threshold}}
and {{dfs.namenode.edit.log.autoroll.check.interval.ms}} to let ANN roll its own log more
frequently in case SBN is stuck on the lock.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message