hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lohit Vijayarenu (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-9985) HDFS Compatible ViewFileSystem
Date Fri, 20 Sep 2013 20:46:54 GMT
Lohit Vijayarenu created HADOOP-9985:
----------------------------------------

             Summary: HDFS Compatible ViewFileSystem
                 Key: HADOOP-9985
                 URL: https://issues.apache.org/jira/browse/HADOOP-9985
             Project: Hadoop Common
          Issue Type: Bug
            Reporter: Lohit Vijayarenu
             Fix For: 2.0.6-alpha


There are multiple scripts and projects like pig, hive, elephantbird refer to HDFS URI as
hdfs://namenodehostport/ or hdfs:/// . In federated namespace this causes problem because
supported scheme for federation is viewfs:// . We will have to force all users to change their
scripts/programs to be able to access federated cluster. 

It would be great if thee was a way to map viewfs scheme to hdfs scheme without exposing it
to users. Opening this JIRA to get inputs from people who have thought about this in their
clusters.

In our clusters we ended up created another class HDFSCompatibleViewFileSystem which hijacks
both hdfs.fs.impl and viewfs.fs.impl and passes down filesystem calls to ViewFileSystem. Is
there any suggested approach other than this?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message