Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 95C11200C79 for ; Fri, 5 May 2017 02:10:10 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 94335160BC4; Fri, 5 May 2017 00:10:10 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id DA2E0160BB0 for ; Fri, 5 May 2017 02:10:09 +0200 (CEST) Received: (qmail 13306 invoked by uid 500); 5 May 2017 00:10:09 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 13294 invoked by uid 99); 5 May 2017 00:10:09 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 May 2017 00:10:08 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 9C91E188A50 for ; Fri, 5 May 2017 00:10:08 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id wZKSOqVhogxI for ; Fri, 5 May 2017 00:10:07 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 114F85FB29 for ; Fri, 5 May 2017 00:10:06 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 4E8ADE0D5F for ; Fri, 5 May 2017 00:10:05 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 861A421DF4 for ; Fri, 5 May 2017 00:10:04 +0000 (UTC) Date: Fri, 5 May 2017 00:10:04 +0000 (UTC) From: "Konstantin Shvachko (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HDFS-7964) Add support for async edit logging MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Fri, 05 May 2017 00:10:10 -0000 [ https://issues.apache.org/jira/browse/HDFS-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-7964: -------------------------------------- Labels: release-blocker (was: ) > Add support for async edit logging > ---------------------------------- > > Key: HDFS-7964 > URL: https://issues.apache.org/jira/browse/HDFS-7964 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: namenode > Affects Versions: 2.0.2-alpha > Reporter: Daryn Sharp > Assignee: Daryn Sharp > Labels: release-blocker > Fix For: 2.8.0, 2.9.0, 3.0.0-alpha1 > > Attachments: HDFS-7964-branch-2.7.patch, HDFS-7964-branch-2.8.0.patch, HDFS-7964.patch, HDFS-7964.patch, HDFS-7964.patch, HDFS-7964.patch, HDFS-7964-rebase.patch > > > Edit logging is a major source of contention within the NN. LogEdit is called within the namespace write log, while logSync is called outside of the lock to allow greater concurrency. The handler thread remains busy until logSync returns to provide the client with a durability guarantee for the response. > Write heavy RPC load and/or slow IO causes handlers to stall in logSync. Although the write lock is not held, readers are limited/starved and the call queue fills. Combining an edit log thread with postponed RPC responses from HADOOP-10300 will provide the same durability guarantee but immediately free up the handlers. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org