hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: Need to add fs shim to use QFS
Date Tue, 09 Oct 2012 19:22:00 GMT
On 5 October 2012 18:27, Thilee Subramaniam <thilee@quantcast.com> wrote:

> We at Quantcast have released QFS 1.0 (Quantcast File System) to open
> source. This is based on the KFS 0.5 (Kosmos Distributed File System),
> a C++ distributed filesystem implementation. KFS plugs into Apache
> Hadoop via the 'kfs' shim that is part of Hadoop codebase.
> QFS has added support for permissions, and also, provides fault tolerance
> through Reed-Solomon encoding as well as replication. There are also a
> number of performance and stability improvements, including a rewrite of
> the client library to allow parallel concurrent I/Os. Going forward, new
> releases of KFS will come from QFS.
Does this mean the kfs plugin can go from the apache tree?

One problem that we've always had with KFS is that nobody ever tested the
filesystem, and it was inevitably out of sync with what was in KFS.

Have you considered just pulling the kfs lib out and releasing the bridge
classes yourself? It's what the other FS suppliers do, as it gives them
more control over the libraries, including the ability to release more


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message