Return-Path: Delivered-To: apmail-hadoop-core-user-archive@www.apache.org Received: (qmail 68302 invoked from network); 13 Apr 2009 02:49:27 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 13 Apr 2009 02:49:27 -0000 Received: (qmail 71946 invoked by uid 500); 13 Apr 2009 02:49:25 -0000 Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org Received: (qmail 71833 invoked by uid 500); 13 Apr 2009 02:49:24 -0000 Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: core-user@hadoop.apache.org Delivered-To: mailing list core-user@hadoop.apache.org Received: (qmail 71823 invoked by uid 99); 13 Apr 2009 02:49:24 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 13 Apr 2009 02:49:24 +0000 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [69.147.107.21] (HELO mrout2-b.corp.re1.yahoo.com) (69.147.107.21) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 13 Apr 2009 02:49:15 +0000 Received: from [1.0.0.0] (proxy7.corp.yahoo.com [216.145.48.98]) by mrout2-b.corp.re1.yahoo.com (8.13.8/8.13.8/y.out) with ESMTP id n3D2m2TA029475 for ; Sun, 12 Apr 2009 19:48:03 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns; h=message-id:date:from:user-agent:mime-version:to:subject: references:in-reply-to:content-type:content-transfer-encoding; b=irnYNliJWXUe4hroCpSlJys+EjlNiVd6nMNKqLgtc355uPvFc9ogSi+V9/+Uh3t/ Message-ID: <49E2A7C0.5080109@yahoo-inc.com> Date: Sun, 12 Apr 2009 19:47:28 -0700 From: Raghu Angadi User-Agent: Thunderbird 2.0.0.21 (X11/20090318) MIME-Version: 1.0 To: core-user@hadoop.apache.org Subject: Re: Changing block size of hadoop References: <384813770904120608w3e570736teeca925ec64be4b8@mail.gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Aaron Kimball wrote: > Blocks already written to HDFS will remain their current size. Blocks are > immutable objects. That procedure would set the size used for all > subsequently-written blocks. I don't think you can change the block size > while the cluster is running, because that would require the NameNode and > DataNodes to re-read their configurations, which they only do at startup. > - Aaron Block size is a client side configuration. NameNode and DataNode don't need to restart. In this particular case, even if client's config is changed, MR may not use the new config for partial set of maps or reducers. Raghu. > On Sun, Apr 12, 2009 at 6:08 AM, Rakhi Khatwani wrote: > >> Hi, >> I would like to know if it is feasbile to change the blocksize of Hadoop >> while map reduce jobs are executing? and if not would the following work? >> 1. >> stop map-reduce 2. stop-hbase 3. stop hadoop 4. change hadoop-sites.xml >> to reduce the blocksize 5. restart all >> whether the data in the hbase tables will be safe and automatically split >> after changing the block size of hadoop?? >> >> Thanks, >> Raakhi >> >