Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D3DCBDBD6 for ; Sun, 17 Mar 2013 18:08:03 +0000 (UTC) Received: (qmail 67070 invoked by uid 500); 17 Mar 2013 18:07:59 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 66986 invoked by uid 500); 17 Mar 2013 18:07:59 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 66979 invoked by uid 99); 17 Mar 2013 18:07:59 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 17 Mar 2013 18:07:59 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of dechouxb@gmail.com designates 209.85.217.177 as permitted sender) Received: from [209.85.217.177] (HELO mail-lb0-f177.google.com) (209.85.217.177) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 17 Mar 2013 18:07:53 +0000 Received: by mail-lb0-f177.google.com with SMTP id go11so4073049lbb.36 for ; Sun, 17 Mar 2013 11:07:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=0jfnqSPpcl76atMhLfN+dggHExJXHP14HPMy+QSzVPc=; b=tLZMqpGPTBYIYfEG5mAGQiWzz6jJUIM19maNtRkD6/gNWknQTbXUfg3dosVmynfaJP WIEzWsuM44oa02RXO0CJwdZMP9J9DZcJKMP1oXGN7oJCGf6NVvFfY7tRUdhtwNiX4YIY eq1ADTYDjLlbn3mRx6iwFpZp2xNUvxVJvwIa+5ra5CLYO3u9u6IzrTmNYvYQGxHOjijA GwNvWNQ2Tn52lK5GWlSs8WG+lLB2dsshGb4Tfh4NWz2c+GZqp5gN6YIBR0PLgJkFc/3M Gzzj46ND7MLnHLNoBpxz45guQ17FRcGy9CyI95YxveXXWSGiZ+zmjzdWDfViw/2L5j4i rCig== MIME-Version: 1.0 X-Received: by 10.112.68.34 with SMTP id s2mr5201301lbt.111.1363543652939; Sun, 17 Mar 2013 11:07:32 -0700 (PDT) Received: by 10.112.135.131 with HTTP; Sun, 17 Mar 2013 11:07:32 -0700 (PDT) In-Reply-To: References: Date: Sun, 17 Mar 2013 19:07:32 +0100 Message-ID: Subject: Re: Replication factor From: Bertrand Dechoux To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=90e6ba30941e11c5b604d822bf1d X-Virus-Checked: Checked by ClamAV on apache.org --90e6ba30941e11c5b604d822bf1d Content-Type: text/plain; charset=ISO-8859-1 The best way would be to test it. The provided links indeed do not seem to help. I would say the default replication factor is the one found in the configuration and it can be overwritten at runtime. I don't remember anything about using the parent directory in order to find the "default replication factor". The use case is understandable though. If the file tree is clean, one could expect many use cases where under one directory each file have the same replication factor but that this replication factor may change from one directory to another. (A ticket could be opened on the jira if none exist about that subject, that way a discussion could start about the interest and difficulty of a patch.) I am afraid you will have to specify the replication factor yourself and/or build yourself a mapping. Regards Bertrand On Tue, Mar 12, 2013 at 10:26 PM, Mohit Anchlia wrote: > Does it mean if I set replication factor on directory /abc and I run a > -put command and add a file to the directory it will use the new > replication factor set on the directory /abc? > > On Tue, Mar 12, 2013 at 2:04 PM, Chris Embree wrote: > >> Aww.. You could've used lmgtfy.com :) >> >> >> On Tue, Mar 12, 2013 at 4:57 PM, varun kumar wrote: >> >>> http://hadoopblogfromvarun.wordpress.com/ >>> >>> >>> On Wed, Mar 13, 2013 at 2:16 AM, Mohit Anchlia wrote: >>> >>>> Is it possible to set replication factor to a different value than the >>>> default at the directory level? >>> >>> >>> >>> >>> -- >>> Regards, >>> Varun Kumar.P >>> >> >> > -- Bertrand Dechoux --90e6ba30941e11c5b604d822bf1d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable The best way would be to test it. The provided links indeed do not seem to = help.

I would say the default replication factor is the one found in= the configuration and it can be overwritten at runtime. I don't rememb= er anything about using the parent directory in order to find the "def= ault replication factor". The use case is understandable though. If th= e file tree is clean, one could expect many use cases where under one direc= tory each file have the same replication factor but that this replication f= actor may change from one directory to another.

(A ticket could be opened on the jira if none exist about that subject,= that way a discussion could start about the interest and difficulty of a p= atch.)

I am afraid you will have to specify the replication factor y= ourself and/or build yourself a mapping.

Regards

Bertrand

On Tue, Mar 1= 2, 2013 at 10:26 PM, Mohit Anchlia <mohitanchlia@gmail.com> wrote:
Does it mean if I set replication factor on = directory /abc and I run a -put command and add a file to the directory it = will use the new replication factor set on the directory /abc?

On Tue, Mar 12, 2013 at 2:04 PM, Chris Embree <= cembree@gmail.com> wrote:
Aww.. =A0You could've used lmgtfy.com :)=20


On Tue, Mar 12, 2013 at 4:57 PM, varun kumar <varun.uid@gmail.com> wrote:
http://hadoopblogfromvarun.wordpress.com= /=20


On Wed, Mar 13, 2013 at 2:16 AM, Mohit Anchlia <= span dir=3D"ltr"><mohitanchlia@gmail.com> wrote:
Is it possible to set replication fac= tor to a different value than the default at the directory level?



--
Regards,
Varun Kumar.P=

=




--
Bertrand De= choux --90e6ba30941e11c5b604d822bf1d--