hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Collins <...@cloudera.com>
Subject Re: Need to add fs shim to use QFS
Date Fri, 05 Oct 2012 17:32:24 GMT
Hey Thilee,

Thanks for contributing. We don't process pull request on the git
mirrors, please upload a patch against trunk and branch-1 if you'd
like this included in Hadoop 1.x and 2.x releases. More info here:


On Fri, Oct 5, 2012 at 10:27 AM, Thilee Subramaniam
<thilee@quantcast.com> wrote:
> We at Quantcast have released QFS 1.0 (Quantcast File System) to open
> source. This is based on the KFS 0.5 (Kosmos Distributed File System),
> a C++ distributed filesystem implementation. KFS plugs into Apache
> Hadoop via the 'kfs' shim that is part of Hadoop codebase.
> QFS has added support for permissions, and also, provides fault tolerance
> through Reed-Solomon encoding as well as replication. There are also a
> number of performance and stability improvements, including a rewrite of
> the client library to allow parallel concurrent I/Os. Going forward, new
> releases of KFS will come from QFS.
> The open source release of QFS is at http://quantcast.github.com/qfs
> QFS plugs into Apache Hadoop the same way KFS does. Currently, one would
> apply the patches or JARs from the QFS source tree onto Apache Hadoop to
> make Hadoop use QFS. The patch for Apache Hadoop 1.0.X can be found at
> https://github.com/quantcast/qfs/blob/master/hadoop/hadoop-1.0.X.patch
> In order to make the integration seamless, we would like to add a 'qfs'
> shim to Apache Hadoop so that the current active branches (1.0.X, 2.X.X,
> 0.23.X) of Apache Hadoop can use QFS.
> Towards this, I've submited an ASF JIRA feature ticket (HADOOP-8885) under
> hadoop-common project, and send a pull request with the QFS shim changes
> to https://github.com/apache/hadoop-common/tree/branch-1.0.2
> I will subsequently submit pull requests to the other active Hadoop
> branches.
> If you have any question, I will be happy yo answer or provide more
> details on QFS.
> - Thilee

View raw message