hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao Sun (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13122) Tailing edits should not update quota counts on ObserverNode
Date Thu, 08 Feb 2018 21:51:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357625#comment-16357625

Chao Sun commented on HDFS-13122:

[~xkrogen]: is this on 2.7.X? since it seems to be already fixed by [HDFS-6763|https://issues-test.apache.org/jira/browse/HDFS-6763].

> Tailing edits should not update quota counts on ObserverNode
> ------------------------------------------------------------
>                 Key: HDFS-13122
>                 URL: https://issues.apache.org/jira/browse/HDFS-13122
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs, namenode
>            Reporter: Erik Krogen
>            Assignee: Erik Krogen
>            Priority: Major
> Currently in {{FSImage#loadEdits()}}, after applying a set of edits, we call
> {code}
> updateCountForQuota(target.getBlockManager().getStoragePolicySuite(), target.dir.rootDir);
> {code}
> to update the quota counts for the entire namespace, which can be very expensive. This
makes sense if we are about to become the ANN, since we need valid quotas, but not on an ObserverNode
which does not need to enforce quotas.
> This is related to increasing the frequency with which the SbNN can tail edits from the
ANN to decrease the lag time for transactions to appear on the Observer.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message