Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D4974D44A for ; Thu, 23 Aug 2012 00:13:43 +0000 (UTC) Received: (qmail 2168 invoked by uid 500); 23 Aug 2012 00:13:43 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 2069 invoked by uid 500); 23 Aug 2012 00:13:43 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 2048 invoked by uid 99); 23 Aug 2012 00:13:43 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Aug 2012 00:13:43 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id B81C72C0911 for ; Thu, 23 Aug 2012 00:13:42 +0000 (UTC) Date: Thu, 23 Aug 2012 11:13:42 +1100 (NCT) From: "Andy Isaacson (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: <1960027189.3451.1345680822754.JavaMail.jiratomcat@arcas> In-Reply-To: <1445695877.36856.1345579838292.JavaMail.jiratomcat@arcas> Subject: [jira] [Updated] (HDFS-3828) Block Scanner rescans blocks too frequently MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Isaacson updated HDFS-3828: -------------------------------- Description: {{BlockPoolSliceScanner#scan}} calls cleanUp every time it's invoked from {{DataBlockScanner#run}} via {{scanBlockPoolSlice}}. But cleanUp unconditionally roll()s the verificationLogs, so after two iterations we have lost the first iteration of block verification times. As a result a cluster with just one block repeatedly rescans it every 10 seconds: {noformat} 2012-08-16 15:59:57,884 INFO datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(391)) - Verification succeeded for BP-2101131164-172.29.122.91-1337906886255:blk_7919273167187535506_4915 2012-08-16 16:00:07,904 INFO datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(391)) - Verification succeeded for BP-2101131164-172.29.122.91-1337906886255:blk_7919273167187535506_4915 2012-08-16 16:00:17,925 INFO datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(391)) - Verification succeeded for BP-2101131164-172.29.122.91-1337906886255:blk_7919273167187535506_4915 {noformat} {quote} To fix this, we need to avoid roll()ing the logs multiple times per period. was: BlockPoolSliceScanner#scan}} calls cleanUp every time it's invoked from {{DataBlockScanner#run}} via {{scanBlockPoolSlice}}. But cleanUp unconditionally {{roll()}}s the verificationLogs, so after two iterations we have lost the first iteration of block verification times. As a result a cluster with just one block repeatedly rescans it every 10 seconds: {noformat} 2012-08-16 15:59:57,884 INFO datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(391)) - Verification succeeded for BP-2101131164-172.29.122.91-1337906886255:blk_7919273167187535506_4915 2012-08-16 16:00:07,904 INFO datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(391)) - Verification succeeded for BP-2101131164-172.29.122.91-1337906886255:blk_7919273167187535506_4915 2012-08-16 16:00:17,925 INFO datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(391)) - Verification succeeded for BP-2101131164-172.29.122.91-1337906886255:blk_7919273167187535506_4915 {noformat} {quote} To fix this, we need to avoid roll()ing the logs multiple times per period. > Block Scanner rescans blocks too frequently > ------------------------------------------- > > Key: HDFS-3828 > URL: https://issues.apache.org/jira/browse/HDFS-3828 > Project: Hadoop HDFS > Issue Type: Bug > Affects Versions: 0.23.0, 2.0.0-alpha > Reporter: Andy Isaacson > Assignee: Andy Isaacson > Attachments: hdfs-3828-1.txt, hdfs3828.txt > > > {{BlockPoolSliceScanner#scan}} calls cleanUp every time it's invoked from {{DataBlockScanner#run}} via {{scanBlockPoolSlice}}. But cleanUp unconditionally roll()s the verificationLogs, so after two iterations we have lost the first iteration of block verification times. As a result a cluster with just one block repeatedly rescans it every 10 seconds: > {noformat} > 2012-08-16 15:59:57,884 INFO datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(391)) - Verification succeeded for BP-2101131164-172.29.122.91-1337906886255:blk_7919273167187535506_4915 > 2012-08-16 16:00:07,904 INFO datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(391)) - Verification succeeded for BP-2101131164-172.29.122.91-1337906886255:blk_7919273167187535506_4915 > 2012-08-16 16:00:17,925 INFO datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(391)) - Verification succeeded for BP-2101131164-172.29.122.91-1337906886255:blk_7919273167187535506_4915 > {noformat} > {quote} > To fix this, we need to avoid roll()ing the logs multiple times per period. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira