ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Veselovsky (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (IGNITE-2195) Accessing from IGFS to HDFS that is in kerberised environment
Date Mon, 08 Feb 2016 16:45:39 GMT

     [ https://issues.apache.org/jira/browse/IGNITE-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ivan Veselovsky updated IGNITE-2195:
------------------------------------
    Assignee: Vladimir Ozerov  (was: Ivan Veselovsky)

> Accessing from IGFS to HDFS that is in kerberised environment
> -------------------------------------------------------------
>
>                 Key: IGNITE-2195
>                 URL: https://issues.apache.org/jira/browse/IGNITE-2195
>             Project: Ignite
>          Issue Type: Bug
>          Components: hadoop, IGFS
>    Affects Versions: ignite-1.4
>            Reporter: Denis Magda
>            Assignee: Vladimir Ozerov
>            Priority: Critical
>              Labels: important
>             Fix For: 1.6
>
>         Attachments: kerbersized_hadoop_fs_factory.zip
>
>
> There is some issue in the current IGFS implementation that doesn't take into account
some Kerberos user related settings which leads to the exception below when there is an attempt
to work with Kerberised cluster
> {noformat}
> Connecting to HDFS with the following settings [uri=null, cfg=all-site.xml, userName=null]
> log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
> org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled.
Available:[TOKEN, KERBEROS]
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2096)
> at org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:944)
> at org.apache.hadoop.hdfs.DistributedFileSystem$DirListingIterator.<init>(DistributedFileSystem.java:927)
> at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:872)
> at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:868)
> at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at org.apache.hadoop.hdfs.DistributedFileSystem.listLocatedStatus(DistributedFileSystem.java:868)
> at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1694)
> at org.apache.hadoop.fs.FileSystem$6.<init>(FileSystem.java:1786)
> at org.apache.hadoop.fs.FileSystem.listFiles(FileSystem.java:1783)
> at com.ig.HadoopFsIssue.main(HadoopFsIssue.java:35)
> Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException):
SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]
> at org.apache.hadoop.ipc.Client.call(Client.java:1427)
> at org.apache.hadoop.ipc.Client.call(Client.java:1358)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
> at com.sun.proxy.$Proxy7.getListing(Unknown Source)
> at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getListing(ClientNamenodeProtocolTranslatorPB.java:573)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
> at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> at com.sun.proxy.$Proxy8.getListing(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.listPaths(DFSClient.java:2094)
> {noformat}
> The issue is fixed in the following way. Need to revisit the fix and check whether it
can lead to some other consequences.
> {noformat}
> /**
> * @return {@link org.apache.hadoop.fs.FileSystem}  instance for this secondary Fs.
> * @throws IOException
> */
> public FileSystem createFileSystem(String userName) throws IOException {
>     userName = IgfsUtils.fixUserName(userName);
>     UserGroupInformation.setConfiguration(cfg);
>     UserGroupInformation ugi = UserGroupInformation.createProxyUser(userName, UserGroupInformation.getCurrentUser());
>     try {
>         return ugi.doAs(new PrivilegedExceptionAction<FileSystem>() {
>             @Override
>             public FileSystem run() throws Exception {
>                     return FileSystem.get(uri, cfg);
>             }
>         });
>     } catch (InterruptedException e) {
>         Thread.currentThread().interrupt();
>         throw new IOException("Failed to create file system due to interrupt.", e);
>     }
> }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message