hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Plamen Jeliazkov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-14059) Test reads from standby on a secure cluster with Configured failover
Date Mon, 12 Nov 2018 18:19:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-14059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16684202#comment-16684202

Plamen Jeliazkov commented on HDFS-14059:

Hey folks,

I seem to have uncovered 2 issues in my testing. Posting them in detail here. If we want to
make sub-tasks out of them let me know and I will do so.


I see the new transitionToObserver operation in `haadmin` but it won’t let me do it. Did
you have to `-forceManual` to get it to go into Observer? I am guessing you guys haven’t
hit this because you did not have automatic failover enabled. Do we want to see this message
or just have it go through?

Here is what I see in console:
[hdfs@*** hadoop-3.3.0-SNAPSHOT]$ bin/hdfs haadmin -transitionToObserver ha2
Automatic failover is enabled for NameNode at ****
Refusing to manually manage HA state, since it may cause
a split-brain scenario or other incorrect state.
If you are very sure you know what you are doing, please
specify the --forcemanual flag.


Seems I am encountering some issue with initializing state in the ObserverReadProxyProvider.
It seems it is expecting some configuration property “hadoop.security.service.user.name.key”
on creation of the DFSClient. My suspicion is that this is coming from making use of HAServiceProtocol’s
getServiceState having the annotation requiring it to be set. I made sure to grab debug logs
showing you the key its looking for and the configuration values.

HAServiceProtocol’s annotation:

This is the error message I get:
2018-11-09 14:53:47,327 DEBUG security.SaslRpcClient: Get token info proto:interface org.apache.hadoop.ha.protocolPB.HAServiceProtocolPB
2018-11-09 14:53:47,327 DEBUG security.SaslRpcClient: tokens aren't supported for this protocol
or user doesn't have one
2018-11-09 14:53:47,327 DEBUG security.SaslRpcClient: Get kerberos info proto:interface org.apache.hadoop.ha.protocolPB.HAServiceProtocolPB
info:@org.apache.hadoop.security.KerberosInfo(clientPrincipal=, serverPrincipal=hadoop.security.service.user.name.key)
2018-11-09 14:53:47,327 DEBUG security.SaslRpcClient: getting serverKey: hadoop.security.service.user.name.key
conf value: null principal: null
java.io.IOException: Couldn't set up IO streams: java.lang.IllegalArgumentException: Failed
to specify server's Kerberos principal name
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:876)
at org.apache.hadoop.ipc.Client$Connection.access$3800(Client.java:421)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1604)
at org.apache.hadoop.ipc.Client.call(Client.java:1435)
at org.apache.hadoop.ipc.Client.call(Client.java:1388)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118)
at com.sun.proxy.$Proxy8.getServiceStatus(Unknown Source)
at org.apache.hadoop.ha.protocolPB.HAServiceProtocolClientSideTranslatorPB.getServiceStatus(HAServiceProtocolClientSideTranslatorPB.java:136)

> Test reads from standby on a secure cluster with Configured failover
> --------------------------------------------------------------------
>                 Key: HDFS-14059
>                 URL: https://issues.apache.org/jira/browse/HDFS-14059
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: test
>            Reporter: Konstantin Shvachko
>            Assignee: Plamen Jeliazkov
>            Priority: Major
> Run standard HDFS tests to verify reading from ObserverNode on a secure HA cluster with

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message