hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Block location in HDFS
Date Wed, 04 Jul 2012 16:09:45 GMT
Right, but do you have a specific 4 nodes in mind when you do the
write? If not, you may as well write to HDFS from a non-DN node and
the block locations will be chosen randomly (with the default policy).
This doesn't _guarantee_ that you get different locations for each
block allocation, but does make the placement random.

Otherwise, I suppose HDFS-385 is what you need, and your placement
class can probably just do a round robin assignment of blocks every

Just curious - why do you seek this?

On Wed, Jul 4, 2012 at 8:31 PM, Giulio D'Ippolito
<giulio.dippolito@gmail.com> wrote:
> Thanks for the answer,
> My problem is slightly different, let say that I have 6 storage point and i
> have 4 files/blocks to insert in the filesystem, i would like that these
> blocks will be placed in 4 different storage points, and of course i would
> like to do the same for all the next 4 blocks.  I'm not interested to the
> replicas for the moment.
>  Thanks
> Giulio
> 2012/7/4 Harsh J <harsh@cloudera.com>
>> With the default replication/block placement policy, all replicas are
>> anyway kept in separate nodes. Is that what you want?
>> Or are you looking to specify a list of locations you want your blocks
>> to go to? For the latter, there's no easy way but you may be
>> interested in https://issues.apache.org/jira/browse/HDFS-385. But you
>> can't send meta-info with each write with hints even with HDFS-385,
>> and may need an external system.
>> On Wed, Jul 4, 2012 at 4:07 PM, Giulio D'Ippolito
>> <giulio.dippolito@gmail.com> wrote:
>> > Hello,
>> >
>> > I would like to insert in the filesystem some blocks, but each of these
>> > blocks must be in a different storage point is there a way to specify
>> > this?
>> >
>> > Thanks
>> --
>> Harsh J

Harsh J

View raw message