hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richards Peter <>
Subject Using apache commons-vfs to read RC files from hdfs
Date Fri, 01 Aug 2014 07:59:39 GMT

I am having a usecase to read files from hdfs and local file system
depending on a configuration parameter. I found that apache commons-vfs
supports various file systems and the latest developer release has an
implementation for hdfs also (though only read support is provided
currently). I find commons-vfs really useful to access txt files from hdfs
and local file system.

However I am not able to access RC Files because commons-vfs exposes only and on FileContent interface to
create readers and writers:

Since the constructors of RCFile.Reader and RCFile.Writer classes do not
accept any InputStreams or OutputStreams, I am not able to access such
files using commons-vfs.

Is it possible to have constructors accepting FSDataInputStream and
FSDataOutputStream as arguments in these classes?

Is there a better way to access files(of any format) from hdfs and local
file system using a common api?


View raw message