ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Denis Magda (JIRA)" <j...@apache.org>
Subject [jira] [Created] (IGNITE-2195) Accessing from IGFS to HDFS that is in kerberised environment
Date Thu, 17 Dec 2015 13:37:46 GMT
Denis Magda created IGNITE-2195:
-----------------------------------

             Summary: Accessing from IGFS to HDFS that is in kerberised environment
                 Key: IGNITE-2195
                 URL: https://issues.apache.org/jira/browse/IGNITE-2195
             Project: Ignite
          Issue Type: Bug
          Components: hadoop, IGFS
    Affects Versions: ignite-1.4
            Reporter: Denis Magda
            Priority: Critical


There is some issue in the current IGFS implementation that doesn't take into account some
Kerberos user related settings which leads to the exception below when there is an attempt
to work with Kerberised cluster

{noformat}
Connecting to HDFS with the following settings [uri=null, cfg=all-site.xml, userName=null]
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled. Available:[TOKEN,
KERBEROS]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2096)
at org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:944)
at org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:927)
at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:872)
at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:868)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.listLocatedStatus(DistributedFileSystem.java:868)
at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1694)
at org.apache.hadoop.fs.FileSystem$6.<init>(FileSystem.java:1786)
at org.apache.hadoop.fs.FileSystem.listFiles(FileSystem.java:1783)
at com.ig.HadoopFsIssue.main(HadoopFsIssue.java:35)
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]
at org.apache.hadoop.ipc.Client.call(Client.java:1427)
at org.apache.hadoop.ipc.Client.call(Client.java:1358)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy7.getListing(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:573)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy8.getListing(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2094)
{noformat}


The issue is fixed in the following way. Need to revisit the fix and check whether it can
lead to some other consequences.

{noformat}
/**
* @return {@link org.apache.hadoop.fs.FileSystem}  instance for this secondary Fs.
* @throws IOException
*/
public FileSystem createFileSystem(String userName) throws IOException {
    userName = IgfsUtils.fixUserName(userName);

    UserGroupInformation.setConfiguration(cfg);
    UserGroupInformation ugi = UserGroupInformation.createProxyUser(userName, UserGroupInformation.getCurrentUser());

    try {
        return ugi.doAs(new PrivilegedExceptionAction<FileSystem>() {
            @Override
            public FileSystem run() throws Exception {
                    return FileSystem.get(uri, cfg);
            }
        });
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();

        throw new IOException("Failed to create file system due to interrupt.", e);
    }
}
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message