hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: How to add another file system in Hadoop
Date Thu, 21 Feb 2013 12:53:13 GMT
What Hemanth points to (fs.TYPE.impl, i.e. fs.cdmi.impl being set to
the classname of the custom FS impl.) is correct for the 1.x releases.
In 2.x and ahead, the class for a URI is auto-discovered from the
classpath (a 'service'). So as long as your jar is present on the
user's runtime, the FS class implementing cdmi:// will get loaded and
work without requiring a config injection globally like 1.x did.

On Thu, Feb 21, 2013 at 6:19 PM, Hemanth Yamijala
<yhemanth@thoughtworks.com> wrote:
> I may be guessing here a bit. Basically a filesystem is identified by the
> protocol part of the URI of a file - so a file on the S3 filesystem will
> have a URI like s3://... If you look at the core-default.xml file in Hadoop
> source, you will see configuration keys like fs.<fs-type>.impl and the value
> is a class that implements that type of filesystem. So, you'd basically have
> to define a cdmi:// type protocol and define a fs.cdmi.impl key in your
> Hadoop configuration.
> Thanks
> hemanth
> On Thu, Feb 21, 2013 at 4:18 PM, Agarwal, Nikhil <Nikhil.Agarwal@netapp.com>
> wrote:
>> Hi,
>> I am planning to add a file system called CDMI under org.apache.hadoop.fs
>> in Hadoop, something similar to KFS or S3 which are already there under
>> org.apache.hadoop.fs. I wanted to ask that say, I write my file system for
>> CDMI and add the package under fs but then how do I tell the core-site.xml
>> or other configuration files to use CDMI file system. Where all do I need to
>> make changes to enable CDMI file system become a part of Hadoop ?
>> Thanks a lot in advance.
>> Regards,
>> Nikhil

Harsh J

View raw message