hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Any samples of how to write a custom FileSystem
Date Wed, 01 Feb 2012 04:37:00 GMT
To write a custom filesystem, extend on the FileSystem class.

Depending on the scheme it is supposed to serve, creating an entry
"fs.<scheme>.impl" in core-site.xml, and then loading it via the
FileSystem.get(URI, conf) API will auto load it for you, provided the
URI you pass has the right scheme.

So supposing I have a FS scheme "foo", I'd register it in core-site.xml as:


And then with a URI object from a path that goes "foo:///mypath/", I'd
do: FileSystem.get(URI, new Configuration()) to get a FooFileSystem

Similarly, if you want to overload the local filesystem with your
class, override the "fs.file.impl" config with your derivative class,
and that'd be used in your configuration loaded programs in future.

A good, not-so-complex impl. example to look at generally would be the

On Wed, Feb 1, 2012 at 9:41 AM, Steve Lewis <lordjoe2000@gmail.com> wrote:
> Specifically how do I register a Custom FileSystem - any sample code
> --
> Steven M. Lewis PhD
> 4221 105th Ave NE
> Kirkland, WA 98033
> 206-384-1340 (cell)
> Skype lordjoe_com

Harsh J
Customer Ops. Engineer
Cloudera | http://tiny.cloudera.com/about

View raw message