hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Any samples of how to write a custom FileSystem
Date Wed, 01 Feb 2012 04:37:00 GMT
To write a custom filesystem, extend on the FileSystem class.

Depending on the scheme it is supposed to serve, creating an entry
"fs.<scheme>.impl" in core-site.xml, and then loading it via the
FileSystem.get(URI, conf) API will auto load it for you, provided the
URI you pass has the right scheme.

So supposing I have a FS scheme "foo", I'd register it in core-site.xml as:

<name>fs.foo.impl</name>
<value>com.myorg.FooFileSystem</value>

And then with a URI object from a path that goes "foo:///mypath/", I'd
do: FileSystem.get(URI, new Configuration()) to get a FooFileSystem
instance.

Similarly, if you want to overload the local filesystem with your
class, override the "fs.file.impl" config with your derivative class,
and that'd be used in your configuration loaded programs in future.

A good, not-so-complex impl. example to look at generally would be the
S3FileSystem.

On Wed, Feb 1, 2012 at 9:41 AM, Steve Lewis <lordjoe2000@gmail.com> wrote:
> Specifically how do I register a Custom FileSystem - any sample code
>
> --
> Steven M. Lewis PhD
> 4221 105th Ave NE
> Kirkland, WA 98033
> 206-384-1340 (cell)
> Skype lordjoe_com



-- 
Harsh J
Customer Ops. Engineer
Cloudera | http://tiny.cloudera.com/about

Mime
View raw message