Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of thilee@quantcast.com designates
 64.78.22.17 as permitted sender)
From: Thilee Subramaniam <thilee@quantcast.com>
To: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>
Date: Fri, 5 Oct 2012 16:27:37 -0700
Subject: Re: Need to add fs shim to use QFS
Thread-Topic: Need to add fs shim to use QFS
Thread-Index: Ac2jUQIWUPFfRC/SSp20w35Jx7Mq0Q==
Message-ID: <CC94BA30.1C98F%thilee@quantcast.com>
In-Reply-To: 
 <CAPn_vTsCpOWDW2kOXkV-x69rW1651P3b=nQXMts6eVFjaHY+Dg@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/14.14.0.111121
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Thank you Eli. I have just uploaded the patch for branch-1 to JIRA. I'll
do the same for trunk soon.

If you have any comments or questions on the patch please let me know.

Thanks,
- Thilee


On 10/5/12 10:32 AM, "Eli Collins" <eli@cloudera.com> wrote:

>Hey Thilee,
>
>Thanks for contributing. We don't process pull request on the git
>mirrors, please upload a patch against trunk and branch-1 if you'd
>like this included in Hadoop 1.x and 2.x releases. More info here:
>http://wiki.apache.org/hadoop/HowToContribute
>
>Thanks,
>Eli
>
>On Fri, Oct 5, 2012 at 10:27 AM, Thilee Subramaniam
><thilee@quantcast.com> wrote:
>> We at Quantcast have released QFS 1.0 (Quantcast File System) to open
>> source. This is based on the KFS 0.5 (Kosmos Distributed File System),
>> a C++ distributed filesystem implementation. KFS plugs into Apache
>> Hadoop via the 'kfs' shim that is part of Hadoop codebase.
>>
>> QFS has added support for permissions, and also, provides fault
>>tolerance
>> through Reed-Solomon encoding as well as replication. There are also a
>> number of performance and stability improvements, including a rewrite of
>> the client library to allow parallel concurrent I/Os. Going forward, new
>> releases of KFS will come from QFS.
>>
>> The open source release of QFS is at http://quantcast.github.com/qfs
>>
>> QFS plugs into Apache Hadoop the same way KFS does. Currently, one would
>> apply the patches or JARs from the QFS source tree onto Apache Hadoop to
>> make Hadoop use QFS. The patch for Apache Hadoop 1.0.X can be found at
>> https://github.com/quantcast/qfs/blob/master/hadoop/hadoop-1.0.X.patch
>>
>> In order to make the integration seamless, we would like to add a 'qfs'
>> shim to Apache Hadoop so that the current active branches (1.0.X, 2.X.X,
>> 0.23.X) of Apache Hadoop can use QFS.
>>
>> Towards this, I've submited an ASF JIRA feature ticket (HADOOP-8885)
>>under
>> hadoop-common project, and send a pull request with the QFS shim changes
>> to https://github.com/apache/hadoop-common/tree/branch-1.0.2
>>
>> I will subsequently submit pull requests to the other active Hadoop
>> branches.
>>
>> If you have any question, I will be happy yo answer or provide more
>> details on QFS.
>>
>> - Thilee
>>