hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Giulio D'Ippolito" <giulio.dippol...@gmail.com>
Subject Re: Block location in HDFS
Date Wed, 04 Jul 2012 19:04:58 GMT
Thanks, you get my problem in your second answer, the placement could be
randomly but must be unique for each block of the subset. I willl
investigate in more details your suggestions. For the privacy of my team
work i will not share this information in a mailing-list, If you like I
willl send you a PM.
Thanks.

Giulio

2012/7/4 Harsh J <harsh@cloudera.com>

> Right, but do you have a specific 4 nodes in mind when you do the
> write? If not, you may as well write to HDFS from a non-DN node and
> the block locations will be chosen randomly (with the default policy).
> This doesn't _guarantee_ that you get different locations for each
> block allocation, but does make the placement random.
>
> Otherwise, I suppose HDFS-385 is what you need, and your placement
> class can probably just do a round robin assignment of blocks every
> time.
>
> Just curious - why do you seek this?
>
> On Wed, Jul 4, 2012 at 8:31 PM, Giulio D'Ippolito
> <giulio.dippolito@gmail.com> wrote:
> > Thanks for the answer,
> >
> > My problem is slightly different, let say that I have 6 storage point
> and i
> > have 4 files/blocks to insert in the filesystem, i would like that these
> > blocks will be placed in 4 different storage points, and of course i
> would
> > like to do the same for all the next 4 blocks.  I'm not interested to the
> > replicas for the moment.
> >  Thanks
> >
> >
> > Giulio
> >
> >
> >
> > 2012/7/4 Harsh J <harsh@cloudera.com>
> >>
> >> With the default replication/block placement policy, all replicas are
> >> anyway kept in separate nodes. Is that what you want?
> >>
> >> Or are you looking to specify a list of locations you want your blocks
> >> to go to? For the latter, there's no easy way but you may be
> >> interested in https://issues.apache.org/jira/browse/HDFS-385. But you
> >> can't send meta-info with each write with hints even with HDFS-385,
> >> and may need an external system.
> >>
> >> On Wed, Jul 4, 2012 at 4:07 PM, Giulio D'Ippolito
> >> <giulio.dippolito@gmail.com> wrote:
> >> > Hello,
> >> >
> >> > I would like to insert in the filesystem some blocks, but each of
> these
> >> > blocks must be in a different storage point is there a way to specify
> >> > this?
> >> >
> >> > Thanks
> >>
> >>
> >>
> >> --
> >> Harsh J
> >
> >
>
>
>
> --
> Harsh J
>

Mime
View raw message