Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of andrew.wang@cloudera.com
 designates 209.85.223.178 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAJqA8gtjLvsQwT6t11e8HCjvjq0uE3Z+MkDdxqTRBnCSZtmDbw@mail.gmail.com>
References: 
 <CAJqA8guR6TtB+SZ_Guuy+jOA=OMPDj32QCrTNP3tSCpM-=AmfQ@mail.gmail.com>
 <CAGB5D2bN1yVoW=qMUJiwYBBvBkzTUAPQicMaERk9uvwemfhBTg@mail.gmail.com>
 <CAJqA8gsH5EYpW44_QUZp+rKx8id03rwdOLzGQcn2=anDY7b1fg@mail.gmail.com>
 <CAGB5D2bd2XivqTqgc2SMABraBZpF6Zu+BJ=ctU5u3WFQPE32tg@mail.gmail.com>
 <CAJqA8gtjLvsQwT6t11e8HCjvjq0uE3Z+MkDdxqTRBnCSZtmDbw@mail.gmail.com>
From: Andrew Wang <andrew.wang@cloudera.com>
Date: Wed, 4 Mar 2015 13:50:53 -0800
Message-ID: 
 <CAGB5D2bKAXfqezOOYGobYzX18sbkPkZ-pFCtrf5e_UChOkt7tg@mail.gmail.com>
Subject: Re: Dose hdfs support the configuration that different blocks can
 have different number of replcias?
To: "hdfs-dev@hadoop.apache.org" <hdfs-dev@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=001a113f998a51939805107d737e

--001a113f998a51939805107d737e
Content-Type: text/plain; charset=UTF-8

Lipeng,

-setrep allows you to change the replication of an existing file. You can
also specify the replication factor when you initially create a file. Not
sure what you mean by "dynamically", that to me means calling setrep.

There is replication or invalidation work done as part of running -setrep.
This is done as a low-priority operation, unless the file is already in a
bad replication state (e.g. under-replicated).

Best,
Andrew

On Wed, Mar 4, 2015 at 12:18 PM, Lipeng Wan <lipengwan86@gmail.com> wrote:

> Hi Andrew,
>
> By using the -setrep command, can we change the replication factor of
> existing files? Or, can we change the replication factor of files
> dynamically? If that's possible, how much data movement overhead will
> occur?
> Thanks!
>
> Lipeng
>
> On Tue, Mar 3, 2015 at 2:57 PM, Andrew Wang <andrew.wang@cloudera.com>
> wrote:
> > Yup, definitely. Check out the -setrep command:
> >
> >
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html#setrep
> >
> > HTH,
> > Andrew
> >
> > On Tue, Mar 3, 2015 at 11:49 AM, Lipeng Wan <lipengwan86@gmail.com>
> wrote:
> >
> >> Hi Andrew,
> >>
> >> Thanks for your reply!
> >> Then is it possible for us to specify different replication factors
> >> for different files?
> >>
> >> Lipeng
> >>
> >> On Tue, Mar 3, 2015 at 2:38 PM, Andrew Wang <andrew.wang@cloudera.com>
> >> wrote:
> >> > Hi Lipeng,
> >> >
> >> > Right now that is unsupported, replication is set on a per-file basis,
> >> not
> >> > per-block.
> >> >
> >> > Andrew
> >> >
> >> > On Tue, Mar 3, 2015 at 11:23 AM, Lipeng Wan <lipengwan86@gmail.com>
> >> wrote:
> >> >
> >> >> Hi devs,
> >> >>
> >> >> By default, hdfs creates same number of replicas for each block. Is
> it
> >> >> possible for us to create more replicas for some of the blocks?
> >> >> Thanks!
> >> >>
> >> >> L. W.
> >> >>
> >>
>

--001a113f998a51939805107d737e--