hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yi Liu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-6871) Improve NameNode performance when creating file
Date Tue, 19 Aug 2014 08:19:19 GMT
Yi Liu created HDFS-6871:

             Summary: Improve NameNode performance when creating file  
                 Key: HDFS-6871
                 URL: https://issues.apache.org/jira/browse/HDFS-6871
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: namenode, performance
            Reporter: Yi Liu
            Assignee: Yi Liu
            Priority: Critical
             Fix For: 2.6.0

Creating file with overwrite flag will cause NN fall into flush edit logs and block other
requests if the file exists.

When we create a file with overwrite flag (default is true) in HDFS, NN will remove original
file if it exists. In FSNamesystem#startFileInternal, NN already holds the write lock, it
calls {{deleteInt}} if the file exists, there is logSync in {{deleteInt}}. So in this case,
logSync is under write lock, it will heavily affect the NN performance. 

We should ignore the force logSync in {{deleteInt}} in this case.

This message was sent by Atlassian JIRA

View raw message