From hdfs-issues-return-13841-apmail-hadoop-hdfs-issues-archive=hadoop.apache.org@hadoop.apache.org Thu Dec 30 01:48:11 2010 Return-Path: Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: (qmail 40828 invoked from network); 30 Dec 2010 01:48:11 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 30 Dec 2010 01:48:11 -0000 Received: (qmail 28807 invoked by uid 500); 30 Dec 2010 01:48:11 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 28780 invoked by uid 500); 30 Dec 2010 01:48:11 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 28772 invoked by uid 99); 30 Dec 2010 01:48:11 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 30 Dec 2010 01:48:11 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.22] (HELO thor.apache.org) (140.211.11.22) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 30 Dec 2010 01:48:08 +0000 Received: from thor (localhost [127.0.0.1]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id oBU1llh5022861 for ; Thu, 30 Dec 2010 01:47:47 GMT Message-ID: <11963412.66921293673667196.JavaMail.jira@thor> Date: Wed, 29 Dec 2010 20:47:47 -0500 (EST) From: "Eli Collins (JIRA)" To: hdfs-issues@hadoop.apache.org Subject: [jira] Updated: (HDFS-1562) Add rack policy tests In-Reply-To: <18885051.66881293673666683.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HDFS-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HDFS-1562: ------------------------------ Description: The existing replication tests (TestBlocksWithNotEnoughRacks, TestPendingReplication, TestOverReplicatedBlocks, TestReplicationPolicy, TestUnderReplicatedBlocks, and TestReplication) are missing tests for rack policy violations. This jira adds the following tests which I created when generating a new patch for HDFS-15. * Test that blocks that have a sufficient number of total replicas, but are not replicated cross rack, get replicated cross rack when a rack becomes available. * Test that new blocks for an underreplicated file will get replicated cross rack. * Mark a block as corrupt, test that when it is re-replicated that it is still replicated across racks. * Reduce the replication factor of a file, making sure that the only block that is across racks is not removed when deleting replicas. * Test that when a block is replicated because a replica is lost due to host failure the the rack policy is preserved. * Test that when the execss replicas of a block are reduced due to a node re-joining the cluster the rack policy is not violated. * Test that rack policy is still respected when blocks are replicated due to node decommissioning. * Test that rack policy is still respected when blocks are replicated due to node decommissioning, even when the blocks are over-replicated. was: The existing replication tests (TestPendingReplication, TestOverReplicatedBlocks, TestReplicationPolicy, TestUnderReplicatedBlocks, TestReplication) are missing tests for rack policy violations. This jira adds the following tests which I created generate a new patch for HDFS-15: * Test that blocks that have a sufficient number of total replicas, but are not replicated cross rack, get replicated cross rack when a rack becomes available. * Test that new blocks for an underreplicated file will get replicated cross rack. * Mark a block as corrupt, test that when it is re-replicated that it is still replicated across racks. * Reduce the replication factor of a file, making sure that the only block that is across racks is not removed when deleting replicas. * Test that when a block is replicated because a replica is lost due to host failure the the rack policy is preserved. * Test that when the execss replicas of a block are reduced due to a node re-joining the cluster the rack policy is not violated. * Test that rack policy is still respected when blocks are replicated due to node decommissioning. * Test that rack policy is still respected when blocks are replicated due to node decommissioning, even when the blocks are over-replicated. > Add rack policy tests > --------------------- > > Key: HDFS-1562 > URL: https://issues.apache.org/jira/browse/HDFS-1562 > Project: Hadoop HDFS > Issue Type: Test > Components: name-node, test > Affects Versions: 0.23.0 > Reporter: Eli Collins > Assignee: Eli Collins > Attachments: hdfs-1562-1.patch > > > The existing replication tests (TestBlocksWithNotEnoughRacks, TestPendingReplication, TestOverReplicatedBlocks, TestReplicationPolicy, TestUnderReplicatedBlocks, and TestReplication) are missing tests for rack policy violations. This jira adds the following tests which I created when generating a new patch for HDFS-15. > * Test that blocks that have a sufficient number of total replicas, but are not replicated cross rack, get replicated cross rack when a rack becomes available. > * Test that new blocks for an underreplicated file will get replicated cross rack. > * Mark a block as corrupt, test that when it is re-replicated that it is still replicated across racks. > * Reduce the replication factor of a file, making sure that the only block that is across racks is not removed when deleting replicas. > * Test that when a block is replicated because a replica is lost due to host failure the the rack policy is preserved. > * Test that when the execss replicas of a block are reduced due to a node re-joining the cluster the rack policy is not violated. > * Test that rack policy is still respected when blocks are replicated due to node decommissioning. > * Test that rack policy is still respected when blocks are replicated due to node decommissioning, even when the blocks are over-replicated. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.