hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8408) MR doesn't work with a non-default ViewFS mount table and security enabled
Date Mon, 21 May 2012 19:59:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280425#comment-13280425
] 

Daryn Sharp commented on HADOOP-8408:
-------------------------------------

Yes, I noticed awhile back that HDFS-3062 was problematic, so it was superseded by my HDFS-3268.
 The handling of tokens for multi-token filesystem is rather funky right now.  HADOOP-7967
and MAPREDUCE-3825 aim to fix the problem.

The way it's intended to work is a filesystem with no tokens is supposed to return null for
its service name.  The {{TokenCache}} used to skip a fs with a null service.  This was problematic
for multi-token filesystems like {{ViewFileSystem}}, so someone changed the {{TokenCache}}
to proceed to acquire tokens from a fs with a null service.  A better approach is to request
tokens from a filesystem: a "normal" fs returns its token if its service is non-null, and
a multi-token filesystem gets the tokens for the leaf fs-es that don't have a null service.
 I hope that makes some sense...
                
> MR doesn't work with a non-default ViewFS mount table and security enabled
> --------------------------------------------------------------------------
>
>                 Key: HADOOP-8408
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8408
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 2.0.0
>            Reporter: Aaron T. Myers
>            Assignee: Aaron T. Myers
>             Fix For: 2.0.1
>
>         Attachments: HDFS-8408.patch
>
>
> With security enabled, if one sets up a ViewFS mount table using the default mount table
name, everything works as expected. However, if you try to create a ViewFS mount table with
a non-default name, you'll end up getting an error like the following (in this case "vfs-cluster"
was the name of the mount table) when running an MR job:
> {noformat}
> java.lang.IllegalArgumentException: java.net.UnknownHostException: vfs-cluster
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message