Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C77A4200BC0 for ; Mon, 31 Oct 2016 21:39:02 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id C6141160B06; Mon, 31 Oct 2016 20:39:02 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 15BFD160AED for ; Mon, 31 Oct 2016 21:39:01 +0100 (CET) Received: (qmail 30395 invoked by uid 500); 31 Oct 2016 20:39:00 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 30108 invoked by uid 99); 31 Oct 2016 20:39:00 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 31 Oct 2016 20:39:00 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id BD8AC2C2A66 for ; Mon, 31 Oct 2016 20:39:00 +0000 (UTC) Date: Mon, 31 Oct 2016 20:39:00 +0000 (UTC) From: "Zhe Zhang (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HDFS-7964) Add support for async edit logging MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 31 Oct 2016 20:39:03 -0000 [ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7964: ---------------------------- Status: Patch Available (was: Reopened) > Add support for async edit logging > ---------------------------------- > > Key: HDFS-7964 > URL: https://issues.apache.org/jira/browse/HDFS-7964 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode > Affects Versions: 2.0.2-alpha > Reporter: Daryn Sharp > Assignee: Daryn Sharp > Fix For: 2.9.0, 3.0.0-alpha1 > > Attachments: HDFS-7964-branch-2.8.0.patch, HDFS-7964-rebase.patch, HDFS-7964.patch, HDFS-7964.patch, HDFS-7964.patch, HDFS-7964.patch > > > Edit logging is a major source of contention within the NN. LogEdit is called within the namespace write log, while logSync is called outside of the lock to allow greater concurrency. The handler thread remains busy until logSync returns to provide the client with a durability guarantee for the response. > Write heavy RPC load and/or slow IO causes handlers to stall in logSync. Although the write lock is not held, readers are limited/starved and the call queue fills. Combining an edit log thread with postponed RPC responses from HADOOP-10300 will provide the same durability guarantee but immediately free up the handlers. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org