hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeffrey E Rodriguez (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14295) Authentication proxy filter on firewall cluster may fail authorization because of getRemoteAddr
Date Wed, 12 Apr 2017 03:28:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15965316#comment-15965316

Jeffrey E  Rodriguez commented on HADOOP-14295:

Thanks for your comments Wei-Chiu Chuang.

I will add some testcases and I've corrected the "Affects version"

Let me explain what it brought up this issue and what my colleagues found. 

When we turn on Hadoop UI Kerberos and try to access Datanode /logs the proxy (Knox) would
get an Authorization failure and it hosts would should as even though Knox wasn't
in local host to Datanode.

We were able to figure out that Datanode have Jetty listening on localhost and that Netty
is used to server request to DataNode, this was a measure to improve performance because of
Netty Async NIO design.

The drawback is that the way Authentication proxy filter uses to figure out the remote server
HttpRequest getRemoteAddr would not work since Netty is a proxy to Knox proxy.

Some of my colleagues suggested to turn use  ChannelHandlerContext.getChannel().getRemoteAddress();
to figure out Knox server host. I think that it is still code on the Netty side and eventually
we would need to set a Header for Jetty to consume. Thus I think it is better to not add the
header on Netty and rely on Knox X-forwarded headers. In any other proxy the solution would
be the same to add the X-forwarded headers.

The impact of this defect on users is that if their Kerberized Hadoop UI, access to Datanode
logs would not work. (you will get a 403). There are workarounds such as being more permisive
on the hostnames from which proxy super user runs.

> Authentication proxy filter on firewall cluster may fail authorization because of getRemoteAddr
> -----------------------------------------------------------------------------------------------
>                 Key: HADOOP-14295
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14295
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: common
>    Affects Versions: 2.7.4, 3.0.0-alpha2, 2.8.1
>            Reporter: Jeffrey E  Rodriguez
>            Assignee: Jeffrey E  Rodriguez
>            Priority: Critical
>             Fix For: 3.0.0-alpha2
>         Attachments: hadoop-14295.001.patch
> Many production environments use firewalls to protect network traffic. In the specific
case of DataNode UI and other Hadoop server for which their ports may fall on the list of
firewalled ports the org.apache.hadoop.security.AuthenticationWithProxyUserFilter user getRemotAdd
(HttpServletRequest) which may return the firewall host such as
> This is unfortunately bad since if you are using a proxy in addition to do perimeter
protection, and you have added your proxy as a super user when  checking for the proxy IP
to authorize user this would fail since getRemoteAdd would return the IP of the firewall (
> "2017-04-08 07:01:23,029 ERROR security.AuthenticationWithProxyUserFilter (AuthenticationWithProxyUserFilter.java:getRemoteUser(94))
- Unable to verify proxy user: Unauthorized connection for super-user: knox from IP"
> I propese to add a check for x-forwarded-for header since proxys usually inject that
header before we do a getRemoteAddr

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message