hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2737) HA: Automatically trigger log rolls periodically on the active NN
Date Sat, 07 Jan 2012 00:41:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13181727#comment-13181727

Todd Lipcon commented on HDFS-2737:

bq. My preference is to read the editlog in progress and leave the rolling of editlog as it
is today, a local NN decision

That's not how it is today -- the SecondaryNameNode is the one that triggers it.

bq. For HA requiring an external trigger for editlog rolling is going back to previous inflexibility

It's not inflexible, since it's not tightly interlocked. The HA SBN may trigger a roll, but
it's also fine if the NN rolls more often than this due to other triggers.
> HA: Automatically trigger log rolls periodically on the active NN
> -----------------------------------------------------------------
>                 Key: HDFS-2737
>                 URL: https://issues.apache.org/jira/browse/HDFS-2737
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ha, name-node
>    Affects Versions: HA branch (HDFS-1623)
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
> Currently, the edit log tailing process can only read finalized log segments. So, if
the active NN is not rolling its logs periodically, the SBN will lag a lot. This also causes
many datanode messages to be queued up in the PendingDatanodeMessage structure.
> To combat this, the active NN needs to roll its logs periodically -- perhaps based on
a time threshold, or perhaps based on a number of transactions. I'm not sure yet whether it's
better to have the NN roll on its own or to have the SBN ask the active NN to roll its logs.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message