Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AE9AEDC44 for ; Wed, 28 Nov 2012 15:22:47 +0000 (UTC) Received: (qmail 27477 invoked by uid 500); 28 Nov 2012 15:22:43 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 27139 invoked by uid 500); 28 Nov 2012 15:22:42 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 27121 invoked by uid 99); 28 Nov 2012 15:22:42 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Nov 2012 15:22:42 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.216.41] (HELO mail-qa0-f41.google.com) (209.85.216.41) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Nov 2012 15:22:35 +0000 Received: by mail-qa0-f41.google.com with SMTP id c26so5417829qad.14 for ; Wed, 28 Nov 2012 07:22:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:content-type:x-gm-message-state; bh=XczlkoUV0jUYLLB5NN6G1e8e1vaamEQS+kCsr3AcUfE=; b=SiDHzKQUHBOMsxZunfrrcmJmGm+QlTwocldyV7qsLIZ0YsdCHMnbh3whRsUsZPM6IF SpLfAVX/b15RSTogvuT0CXs0CJaTG8LntL/eRcgewiKKIxjVBxecPyT9RZhYbnjqrc/C +u9Q/eMyQ5cL4ldoxjfwT4BBM2dgDqUKZz1Xi46vbn3BgYlDTYuxHk0e34AoeZ5eU5jH 3JWEjXydj5GiQbnxamJjVz2wkuzVOe2Ll+yozZ0HN3d3rfP7sNTTLeuZozatBoICSOKW cIJ7D00E5TghgeFUlgQhmF1NGveix2Q+1WTyf7i0xFVoK/9q+7f/9Ph+kbvS606htbqV nOaA== MIME-Version: 1.0 Received: by 10.49.30.34 with SMTP id p2mr23854445qeh.15.1354116134280; Wed, 28 Nov 2012 07:22:14 -0800 (PST) Received: by 10.49.120.134 with HTTP; Wed, 28 Nov 2012 07:22:14 -0800 (PST) X-Originating-IP: [12.186.100.34] In-Reply-To: References: Date: Wed, 28 Nov 2012 07:22:14 -0800 Message-ID: Subject: Re: Replacing a hard drive on a slave From: Mark Kerzner To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=047d7bdc8d262b1da904cf8fbbe3 X-Gm-Message-State: ALoCoQkv6IVuH+pl+nKc5W5ZbPHcwPR7a5cyQaLFkkGaILkujAGx3ns4AeR79mS58tyu2Riy9mU+ X-Virus-Checked: Checked by ClamAV on apache.org --047d7bdc8d262b1da904cf8fbbe3 Content-Type: text/plain; charset=ISO-8859-1 What happens if I stop the datanode, miss the 10 min 30 seconds deadline, and restart the datanode say 30 minutes later? Will Hadoop re-use the data on this datanode, balancing it with HDFS? What happens to those blocks that correspond to file that have been updated meanwhile? Mark On Wed, Nov 28, 2012 at 6:51 AM, Stephen Fritz wrote: > HDFS will not start re-replicating blocks from a dead DN for 10 minutes 30 > seconds by default. > > Right now there isn't a good way to replace a disk out from under a > running datanode, so the best way is: > - Stop the DN > - Replace the disk > - Restart the DN > > > > > On Wed, Nov 28, 2012 at 9:14 AM, Mark Kerzner wrote: > >> Hi, >> >> can I remove one hard drive from a slave but tell Hadoop not to replicate >> missing blocks for a few minutes, because I will return it back? Or will >> this not work at all, and will Hadoop continue replicating, since I removed >> blocks, even for a short time? >> >> Thank you. Sincerely, >> Mark >> > > --047d7bdc8d262b1da904cf8fbbe3 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable What happens if I stop the datanode, miss the 10 min 30 seconds deadline, a= nd restart the datanode say 30 minutes later? Will Hadoop re-use the data o= n this datanode, balancing it with HDFS? What happens to those blocks that = correspond to file that have been updated meanwhile?

Mark

On Wed, Nov 28, 2012 at 6:51 AM,= Stephen Fritz <stephenf@cloudera.com> wrote:
HDFS will not start re-replicating blocks from a dead DN for 10 minutes 30 = seconds by default.

Right now there isn't a good way to replace = a disk out from under a running datanode, so the best way is:
- Stop the= DN
- Replace the disk
- Restart the DN




On Wed, Nov 28, 2012 at 9:14 = AM, Mark Kerzner <mark.kerzner@shmsoft.com> wrote:
Hi,

can I remove one hard drive from = a slave but tell Hadoop not to replicate missing blocks for a few minutes, = because I will return it back? Or will this not work at all, and will Hadoo= p continue replicating, since I removed blocks, even for a short time?

Thank you. Sincerely,
Mark


--047d7bdc8d262b1da904cf8fbbe3--