hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From deepya <m_dee...@yahoo.co.in>
Subject Re: a doubt regarding an appropriate file system
Date Mon, 30 Mar 2009 07:46:23 GMT


   Can you please specify in detail what kind of problems I will face if I
use Hadoop for this project.


TimRobertson100 wrote:
> I believe Hadoop is not best suited to many small files like yours but
> is really geared to handling very large files that get split into many
> smaller files (like 128M chunks) and HDFS is designed with this in
> mind.  Therefore I could *imagine* that there are other distributed
> file systems that would far outperform HDFS if they were designed to
> replicate and track small files without any *split* and *merging*
> which Hadoop provides.
> Having not used MogileFS I cant really advise well but a quick read
> through does look like it might be a candidate for you to consider -
> it looks like it distributes across machines and tracks replicas like
> HDFS without the splitting, and offers access through http to the
> individual files which I could imagine would be ideal for pulling back
> small images.
> Please don't just follow my advise though - I am still a relative
> newbie to DFS's in general.
> Cheers
> Tim
> On Sun, Mar 29, 2009 at 12:51 PM, deepya <m_deepya@yahoo.co.in> wrote:
>> Hi,
>>  I am doing a project scalable storage server to store images.Can Hadoop
>> efficiently support this purpose???Our image size will be around 250 to
>> 300
>> KB each.But we have many such images.Like the total storage may run upto
>> petabytes( in future) .At present it is in gigabytes.
>>   We want to access these images via apache server.I mean,is there any
>> mechanism that we can directly talk to hdfs via apache server???
>> I went through one of the posts here and got to know that rather than
>> using
>> FUSE it is better to use HDFS API.That is fine.But they also mentioned
>> that
>> mozilefs will be more appropriate.
>> Can some one please clarify why mozilefs is more appropriate.Cant hadoop
>> be
>> used???How is mozile more advantageous.Can you suggest which filesystem
>> would be more appropriate for the project I am doing at present.
>> Thanks in advance
>> SreeDeepya
>> --
>> View this message in context:
>> http://www.nabble.com/a-doubt-regarding-an-appropriate-file-system-tp22766331p22766331.html
>> Sent from the Hadoop core-user mailing list archive at Nabble.com.

View this message in context: http://www.nabble.com/a-doubt-regarding-an-appropriate-file-system-tp22766331p22777879.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.

View raw message