hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charles Woerner <charleswoer...@gmail.com>
Subject Re: Hadoop vs Ceph and GlusterFS
Date Sat, 28 Dec 2013 19:52:54 GMT
May i also suggest benchmarking against qfs (the Quantcast FileSystem, formerly kosmosfs)?
 

https://github.com/quantcast/qfs

It's a high performance native (c++) drop in replacement for hdfs and works extremely well
with hadoop.  It ships with hdfs java api bindings as well as native bindings for a number
of dynamic languages, including python.

One of the main strengths of qfs over the other filesystems you are evaluating is that it
makes use of configurable Reed-Solomon encoding to improve the storage-to-fault tolerance
relationship while maintaining best of class performance.  It also supports appends.  

With respect to setup, it's
very easy to build and install.

(I work for Quantcast and we use qfs exclusively for these reasons.)

Sent from my iPhone

> On Dec 28, 2013, at 11:26 AM, Kurt Moesky <kurtmoesky@gmail.com> wrote:
> 
> Hi Charles,
> 
> That is actually what we're doing, comparing the Hadoop file system to Ceph and GlusterFS.
Just looking for some input from the field as that what you experts see as the strengths of
HDFS over Ceph and GlusterFS.
> 
> Thanks,
> Kurt
> 
> 
>> On Sat, Dec 28, 2013 at 11:42 AM, Charles Earl <charles.cearl@gmail.com> wrote:
>> Would it not be better to compare HDFS as the others are distributed file systems?
>> Charles
>> 
>> On Dec 28, 2013, at 1:40 PM, Kurt Moesky <kurtmoesky@gmail.com> wrote:
>> 
>> > Hi guys,
>> >
>> > I am working on a write-up of Hadoop, Ceph and GlusterFS and was wondering if
you could chime in with some benefits of Hadoop over the other two?
>> >
>> > I know Hadoop is widely used by the likes of Yahoo and Facebook.
>> >
>> > Are there benefits in scaling, management (I like the Ambari interface) etc?
>> >
>> > Thanks.
> 

Mime
View raw message