hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: How to make command setrepr take effect on data node ?
Date Sun, 13 Jun 2010 18:17:41 GMT
On Sat, Jun 12, 2010 at 8:18 PM, Jeff Zhang <zjffdu@gmail.com> wrote:

> So when will the replication happen ? Do you know which part of code
> control the replication ? Thanks.
>
>
It's all throughout FSNamesystem and related classes. I wouldn't recommend
digging in there unless you have a couple days to kill - it gets quite
hairy! :)

Replication should happen close to immediately unless there is already a
large queue of underreplicated blocks. Replication happens through a
priority queue, where the priority is based on how far under-replicated a
block is.

-Todd


>
> On Sun, Jun 13, 2010 at 11:10 AM, Eric Sammer <esammer@cloudera.com>
> wrote:
> > There is no way to force replication other than setrep. Do you have
> > fewer nodes than configured replicas? Hadoop will not place two
> > replicas on one data node, for instance.
> >
> > On Sat, Jun 12, 2010 at 12:18 AM, Jeff Zhang <zjffdu@gmail.com> wrote:
> >> But when I invoke command fsck, it show that one replica is missing.
> >> So how can I force the replication ?
> >>
> >> On Sat, Jun 12, 2010 at 12:13 PM, Eric Sammer <esammer@cloudera.com>
> wrote:
> >>> Jeff:
> >>>
> >>> The name node will instruct the data nodes to do either remove or
> >>> create new replicas. There's nothing additional you need to do other
> >>> than fs -setrep.
> >>>
> >>> On Fri, Jun 11, 2010 at 3:18 AM, Jeff Zhang <zjffdu@gmail.com> wrote:
> >>>> Hi all,
> >>>>
> >>>> The command bin/hadoop setrepr only affect the metadata in namenode.
> >>>> So How can I make it take effect on data node ? Thanks.
> >>>>
> >>>> --
> >>>> Best Regards
> >>>>
> >>>> Jeff Zhang
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Eric Sammer
> >>> phone: +1-917-287-2675
> >>> twitter: esammer
> >>> data: www.cloudera.com
> >>>
> >>
> >>
> >>
> >> --
> >> Best Regards
> >>
> >> Jeff Zhang
> >>
> >
> >
> >
> > --
> > Eric Sammer
> > phone: +1-917-287-2675
> > twitter: esammer
> > data: www.cloudera.com
> >
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
View raw message