Return-Path: 
 <user-return-2107-apmail-hadoop-common-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-common-user-archive@www.apache.org
Delivered-To: apmail-hadoop-common-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id C544BD62B
	for <apmail-hadoop-common-user-archive@www.apache.org>;
 Mon, 15 Oct 2012 20:57:43 +0000 (UTC)
Received: (qmail 74168 invoked by uid 500); 15 Oct 2012 20:57:39 -0000
Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org
Received: (qmail 73980 invoked by uid 500); 15 Oct 2012 20:57:39 -0000
Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:user@hadoop.apache.org>
List-Id: <user.hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Delivered-To: mailing list user@hadoop.apache.org
Received: (qmail 73973 invoked by uid 99); 15 Oct 2012 20:57:39 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 15 Oct 2012 20:57:39 +0000
X-ASF-Spam-Status: No, hits=-0.7 required=5.0
	tests=RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of silvianhadoop@gmail.com
 designates 74.125.82.176 as permitted sender)
Received: from [74.125.82.176] (HELO mail-we0-f176.google.com) (74.125.82.176)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 15 Oct 2012 20:57:34 +0000
Received: by mail-we0-f176.google.com with SMTP id z53so3810832wey.35
        for <user@hadoop.apache.org>; Mon, 15 Oct 2012 13:57:13 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=WYV5qsL703DEM73xFsTHVVF1KYWTcq3fVk/a/HkEi4I=;
        b=jKEaQdxLWS/eBfoIlEFJgdfT1pPXK9hQYImVjW8brhPnkbCJ5+oQKPha1897PDEm7J
         4jjGqVqOceDByUctkyaidcUEVHynIcmFN2UsWjIdDlyBmnFKbWULXni9MD5ZDW15VxUf
         aGkbUiADEJKn0Sc3mc812xL+H2gAoOUddq2E8yUotNUWvHMtORb2HJCaC28em5wyptHk
         aUxqZC3bqN3OGuat3W3/wj7YMjaqr57zv2GShdZEEThQNbdWVmFBUmiZb3TTbKwLjvV8
         oGgV0FzV+PRfZCgoQIOG0Ka9OIA+K6z2HT3n9w3+yt0vW35WhgRRMcfqPtmW/L/eYbu2
         ASnw==
MIME-Version: 1.0
Received: by 10.216.217.195 with SMTP id i45mr7927171wep.14.1350334633152;
 Mon, 15 Oct 2012 13:57:13 -0700 (PDT)
Received: by 10.194.21.228 with HTTP; Mon, 15 Oct 2012 13:57:13 -0700 (PDT)
In-Reply-To: 
 <CABCYYb9yOjYH9CKGJ3BeR+JB_w_ZfpQUX8pBN+9q6834U0pyrA@mail.gmail.com>
References: 
 <CAMWNowH6az+SHesSrYn5zdcx5DsqX9Ra3a9XTbUi-xU53SXrEQ@mail.gmail.com>
	<CABCYYb8Qn6rTKqRJqau4AWtpAM0JFzUDMPYEJUPGMh_VLOBrqQ@mail.gmail.com>
	<CAOcnVr1bYgtvpWVqsqjM4sAOuTav3f6vtY2sb1v45b4TUOXBew@mail.gmail.com>
	<CABCYYb9yOjYH9CKGJ3BeR+JB_w_ZfpQUX8pBN+9q6834U0pyrA@mail.gmail.com>
Date: Mon, 15 Oct 2012 13:57:13 -0700
Message-ID: 
 <CAMWNowH_sT26iVvRPF0_rZ2-qbOyBReGPFC7cBhqLfvu63-mMA@mail.gmail.com>
Subject: Re: final the dfs.replication and fsck
From: Patai Sangbutsarakum <silvianhadoop@gmail.com>
To: user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
X-Virus-Checked: Checked by ClamAV on apache.org

Thanks Harsh, dfs.replication.max does do the magic!!

On Mon, Oct 15, 2012 at 1:19 PM, Chris Nauroth <cnauroth@hortonworks.com> wrote:
> Thank you, Harsh.  I did not know about dfs.replication.max.
>
>
> On Mon, Oct 15, 2012 at 12:23 PM, Harsh J <harsh@cloudera.com> wrote:
>>
>> Hey Chris,
>>
>> The dfs.replication param is an exception to the <final> config
>> feature. If one uses the FileSystem API, one can pass in any short
>> value they want the replication to be. This bypasses the
>> configuration, and the configuration (being per-file) is also client
>> sided.
>>
>> The right way for an administrator to enforce a "max" replication
>> value at a create/setRep level, would be to set
>> the dfs.replication.max to a desired value at the NameNode and restart
>> it.
>>
>> On Tue, Oct 16, 2012 at 12:48 AM, Chris Nauroth
>> <cnauroth@hortonworks.com> wrote:
>> > Hello Patai,
>> >
>> > Has your configuration file change been copied to all nodes in the
>> > cluster?
>> >
>> > Are there applications connecting from outside of the cluster?  If so,
>> > then
>> > those clients could have separate configuration files or code setting
>> > dfs.replication (and other configuration properties).  These would not
>> > be
>> > limited by final declarations in the cluster's configuration files.
>> > <final>true</final> controls configuration file resource loading, but it
>> > does not necessarily block different nodes or different applications
>> > from
>> > running with completely different configurations.
>> >
>> > Hope this helps,
>> > --Chris
>> >
>> >
>> > On Mon, Oct 15, 2012 at 12:01 PM, Patai Sangbutsarakum
>> > <silvianhadoop@gmail.com> wrote:
>> >>
>> >> Hi Hadoopers,
>> >>
>> >> I have
>> >> <property>
>> >>     <name>dfs.replication</name>
>> >>     <value>2</value>
>> >>     <final>true</final>
>> >>   </property>
>> >>
>> >> set in hdfs-site.xml in staging environment cluster. while the staging
>> >> cluster is running the code that will later be deployed in production,
>> >> those code is trying to have dfs.replication of 3, 10, 50, other than
>> >> 2; the number that developer thought that will fit in production
>> >> environment.
>> >>
>> >> Even though I final the property dfs.replication in staging cluster
>> >> already. every time i run fsck on the staging cluster i still see it
>> >> said under replication.
>> >> I thought final keyword will not honor value in job config, but it
>> >> doesn't seem so when i run fsck.
>> >>
>> >> I am on cdh3u4.
>> >>
>> >> please suggest.
>> >> Patai
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>
>