hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Franck Besnard <besn0...@yahoo.com>
Subject Using HDFS to store few MB files for file sharing purposes
Date Thu, 10 Nov 2011 12:52:50 GMT

I'm looking at HDFS as a potential canditate to store files shared by few hundreds of users.
This idea would be to mount local drive directly linked to the HDFS cloud. The datanodes would
be deployed on each desktops which means up to 500 DNs.

Now that said browsing the web it seems that :
- HDFS is design to store large files (block at a minimum of 64Mb)
>- HDFS has a SPOC with the NN
>- HDFS security seems to be very limited
In such a situation, the average file size would be 500KB. So i would think of using a 1MB

Does anybody have experience running such a configuration ? Feedback ?
Any opinion ? Should i compltely drop that idea ?
Would the NN accept 500 mounts through FUSE ?

Thx & brgds

View raw message