hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-7964) Add support for async edit logging
Date Fri, 20 Mar 2015 21:30:38 GMT
Daryn Sharp created HDFS-7964:

             Summary: Add support for async edit logging
                 Key: HDFS-7964
                 URL: https://issues.apache.org/jira/browse/HDFS-7964
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: namenode
    Affects Versions: 2.0.2-alpha
            Reporter: Daryn Sharp
            Assignee: Daryn Sharp

Edit logging is a major source of contention within the NN.  LogEdit is called within the
namespace write log, while logSync is called outside of the lock to allow greater concurrency.
 The handler thread remains busy until logSync returns to provide the client with a durability
guarantee for the response.

Write heavy RPC load and/or slow IO causes handlers to stall in logSync.  Although the write
lock is not held, readers are limited/starved and the call queue fills.  Combining an edit
log thread with postponed RPC responses from HADOOP-9953 will provide the same durability
guarantee but immediately free up the handlers.

This message was sent by Atlassian JIRA

View raw message