From hdfs-issues-return-75258-apmail-hadoop-hdfs-issues-archive=hadoop.apache.org@hadoop.apache.org Fri Jan 3 02:40:50 2014 Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DE35C1027C for ; Fri, 3 Jan 2014 02:40:50 +0000 (UTC) Received: (qmail 11964 invoked by uid 500); 3 Jan 2014 02:40:50 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 11936 invoked by uid 500); 3 Jan 2014 02:40:50 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 11927 invoked by uid 99); 3 Jan 2014 02:40:50 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 03 Jan 2014 02:40:50 +0000 Date: Fri, 3 Jan 2014 02:40:50 +0000 (UTC) From: "Andrew Wang (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HDFS-5589) Namenode loops caching and uncaching when data should be uncached MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-5589: ------------------------------ Attachment: hdfs-5589-1.patch Patch attached. The cache/uncache bug was that we weren't clearing out blocks that weren't marked during the directive scan. So, an orphan block would retain the old mark and replication factor, and become cached again on the next rescan when the mark flipped to the old value. I also incorporated HDFS-5507 (considering stale and capacity when caching), and also fixed another bug I found where we'd try to cache a block again on a node that already had it cached. > Namenode loops caching and uncaching when data should be uncached > ----------------------------------------------------------------- > > Key: HDFS-5589 > URL: https://issues.apache.org/jira/browse/HDFS-5589 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: caching, namenode > Affects Versions: 3.0.0 > Reporter: Andrew Wang > Attachments: hdfs-5589-1.patch > > > This was reported by [~cnauroth] and [~brandonli], and [~schu] repro'd it too. > If you add a new caching directive then remove it, the Namenode will sometimes get stuck in a loop where it sends DNA_CACHE and then DNA_UNCACHE repeatedly to the datanodes where the data was previously cached. -- This message was sent by Atlassian JIRA (v6.1.5#6160)