hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Using HDFS to store few MB files for file sharing purposes
Date Thu, 10 Nov 2011 21:30:05 GMT
Hi Franck,

This is not HDFS's intended use case and I do not think it will solve
your problem very successfully.

You're probably better off looking at some other technologies.


On Thu, Nov 10, 2011 at 4:52 AM, Franck Besnard <besn0847@yahoo.com> wrote:
> Hello,
> I'm looking at HDFS as a potential canditate to store files shared by few
> hundreds of users.
> This idea would be to mount local drive directly linked to the HDFS cloud.
> The datanodes would be deployed on each desktops which means up to 500 DNs.
> Now that said browsing the web it seems that :
> - HDFS is design to store large files (block at a minimum of 64Mb)
> - HDFS has a SPOC with the NN
> - HDFS security seems to be very limited
> In such a situation, the average file size would be 500KB. So i would think
> of using a 1MB block.
> Does anybody have experience running such a configuration ? Feedback ?
> Any opinion ? Should i compltely drop that idea ?
> Would the NN accept 500 mounts through FUSE ?
> Thx & brgds

Todd Lipcon
Software Engineer, Cloudera

View raw message