hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9985) HDFS Compatible ViewFileSystem
Date Mon, 23 Sep 2013 22:09:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13775710#comment-13775710
] 

Andrew Wang commented on HADOOP-9985:
-------------------------------------

Hey Lohit,

Why not use the default FS config parameter (fs.default.name) instead of using fully-qualified
URLs? Then you just need to change the client config when switching clusters.
                
> HDFS Compatible ViewFileSystem
> ------------------------------
>
>                 Key: HADOOP-9985
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9985
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Lohit Vijayarenu
>             Fix For: 2.0.6-alpha
>
>
> There are multiple scripts and projects like pig, hive, elephantbird refer to HDFS URI
as hdfs://namenodehostport/ or hdfs:/// . In federated namespace this causes problem because
supported scheme for federation is viewfs:// . We will have to force all users to change their
scripts/programs to be able to access federated cluster. 
> It would be great if thee was a way to map viewfs scheme to hdfs scheme without exposing
it to users. Opening this JIRA to get inputs from people who have thought about this in their
clusters.
> In our clusters we ended up created another class HDFSCompatibleViewFileSystem which
hijacks both hdfs.fs.impl and viewfs.fs.impl and passes down filesystem calls to ViewFileSystem.
Is there any suggested approach other than this?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message