nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "WaterslideKid (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (NIFI-4273) incorrect data access urls when using site to site when connecting through reverse proxies. Inability for site to site data access urls to utilize proxied user identites.
Date Tue, 08 Aug 2017 15:30:00 GMT

     [ https://issues.apache.org/jira/browse/NIFI-4273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

WaterslideKid updated NIFI-4273:
--------------------------------
    Description: 
Issue:
   Data access using site-to-site binding is based on local environment and does not easily
support a reverse proxy configuration.  The UI works great under a reverse proxy configuration,
but a site to site configuration fails.  It appears that the underlying site to site data
access URLs are built from local environment information rather than configuration and do
not honor the SSL X-ProxiedEntitiesChain or other reverse proxy context information to build
proper URLs.

My configuration:
I have two nifi systems that I want to connect to each other via NIFI site to site.  One is
monitoring logs and intending to publish these logs by offering a site to site web service/server
in order to be collected by a client nifi.  My desire is to have the client nifi contact the
server to pull logs by connecting through a nginx reverse proxy.  All of this is done over
SSL using mutual SSL.  I'm able to get these systems talking to each over directly but having
issues talking through a reverse proxy.

In my preferred configuration, the nifi server behind the reverse proxy is bound to localhost
on port 8443.  The reverse proxy is bound to an external ip address on port 4443.  The client
is set up to talk to the server using site to iste over the reverse proxy on the external
ip on port 4443.  In the UI, the configuration is able to view the flow ports for site to
site but data refuses to flow.  The client complains that it is unable to talk with the data
channel via site to site to the server because it is unable to connect to <hostname>:8443
- which is incorrect.  It appears that the site to site data channel gets its configuration
by choosing the hostname and port of the nifi service rather than obeying the reverse proxy
configuration of the UI.  There is no way to configure the client or server to use an alternative
port or hostname for site to site data.    

   I have a successful configuration of a mutual SSL enabled NIFI reverse proxy that allows
UI clients to access a NIFI server and pass on client SSL information.

   Here is my site to site configuration


{code:java}
# --- the following is a snipplet from an nginx configuration
listen 4443 ssl default_server;
ssl_certificate      /etc/nginx/cert/nifi/server.pem;
ssl_certificate_key  /etc/nginx/cert/nifi/server.key;

      #require clients to authenticate with certs that are part of CA
      #force mutual SSL
      ssl_client_certificate /etc/nginx/cert/nifi/ca.pem;
      ssl_verify_client on;

      ssl on;

	 location / {
                #give the reverse proxy server a client certificate for nifi
                proxy_ssl_certificate /etc/nginx/cert/nifi/rproxy.pem;
                proxy_ssl_certificate_key /etc/nginx/cert/nifi/rproxy.key;
                
                #note that you have to give this client the "proxy user requests"
                #access policy so that it can pass on clients using the
                # X-ProxiedEntitiesChain header -- only valid under mutual ssl

                #do some other proxy optimization
                proxy_buffering off;

                #the following are special headers needed by nifi to build proper urls
                proxy_set_header X-ProxyScheme https;
                proxy_set_header X-ProxyHost $host; 
                proxy_set_header X-ProxyPort $server_port;
                proxy_set_header X-ProxiedEntitiesChain "<${ssl_client_s_dn}>";

                #talk to nifi server bound to localhost
                proxy_pass https://127.0.0.1:8443;
        }
#-- end sniplet
{code}

Attempt #2 -
I'm able to configure the nginx to listen on 8443 when bound to the external ip and have the
nifi service bound to 8443 on the localhost/loopback interface- this configuraiton allows
the URLS provided over site to site will work but another issue arrises.  I get a wierd error
on the client that states:

o.a.nifi.remote.StandardRemoteGroupPort RemoteGroupPort[name=nifiout,targets=https://rproxy:8443/nifi]
failed to communicate with https://rproxy:8443/nifi due to org.apache.nifi.remote.exception.HandshakeException:
StandardRootGroupPort[id=<redacted>] authorization failed for user <redacted>
because Unable to modify data transfers to Output Port with ID <redacted>. Contact the
system administrator.

What I have been able to determine is that the site to site data access service is not able
to handle the X-ProxiedEntitiesChain to get access the data api.

Attempt #3 -
I can get site to site working through a reverse proxy by tricking out the nifi systems to
think they are talking directly to each other.  In this configuration I bind the nifi to localhost:8443,
bind the reverse proxy on the same host to externalIP:8443 and replicate the nifi client certificates
on the nifi endpoint to the nginx proxy.  The nifi client connect through the reverse proxy
and the reverse proxy pretends to be the nifi client to talk to the localhost server.  This
works.

This looks like:

      #--- the following is a snipplet from an nginx configuration
	 listen $externalip:8443 ssl default_server;
	 ssl_certificate      /etc/nginx/cert/nifi/server.pem;
      ssl_certificate_key  /etc/nginx/cert/nifi/server.key;

      #require clients to authenticate with certs that are part of CA
      #force mutual SSL
      ssl_client_certificate /etc/nginx/cert/nifi/ca.pem;
      ssl_verify_client on;

      ssl on;

	 location / {
                #give the reverse proxy server a client certificate for nifi
                proxy_ssl_certificate /etc/nginx/cert/nifi/nifi_client.pem;
                proxy_ssl_certificate_key /etc/nginx/cert/nifi/nifi_client.key;
                #this is replicated from the connecting nifi-client
		 
                #do some other proxy optimization
                proxy_buffering off;

                #talk to nifi server bound to localhost
                proxy_pass https://127.0.0.1:8443;
        }
	   #-- end sniplet

I am able to trick out a reverse proxy in order to get site to site working, but would prefer
a configuration option that supports site to site data access through an alternative reverse
proxy hostname and port using either a server or client configuration alternative.  Currently
the site to site proxy configuration is based on the assumption of a client proxy rather than
a server-site reverse proxy.


  was:
Issue:
   Data access using site-to-site binding is based on local environment and does not easily
support a reverse proxy configuration.  The UI works great under a reverse proxy configuration,
but a site to site configuration fails.  It appears that the underlying site to site data
access URLs are built from local environment information rather than configuration and do
not honor the SSL X-ProxiedEntitiesChain or other reverse proxy context information to build
proper URLs.

My configuration:
I have two nifi systems that I want to connect to each other via NIFI site to site.  One is
monitoring logs and intending to publish these logs by offering a site to site web service/server
in order to be collected by a client nifi.  My desire is to have the client nifi contact the
server to pull logs by connecting through a nginx reverse proxy.  All of this is done over
SSL using mutual SSL.  I'm able to get these systems talking to each over directly but having
issues talking through a reverse proxy.

In my preferred configuration, the nifi server behind the reverse proxy is bound to localhost
on port 8443.  The reverse proxy is bound to an external ip address on port 4443.  The client
is set up to talk to the server using site to iste over the reverse proxy on the external
ip on port 4443.  In the UI, the configuration is able to view the flow ports for site to
site but data refuses to flow.  The client complains that it is unable to talk with the data
channel via site to site to the server because it is unable to connect to <hostname>:8443
- which is incorrect.  It appears that the site to site data channel gets its configuration
by choosing the hostname and port of the nifi service rather than obeying the reverse proxy
configuration of the UI.  There is no way to configure the client or server to use an alternative
port or hostname for site to site data.    

   I have a successful configuration of a mutual SSL enabled NIFI reverse proxy that allows
UI clients to access a NIFI server and pass on client SSL information.

   Here is my site to site configuration

# --- the following is a snipplet from an nginx configuration
listen 4443 ssl default_server;
ssl_certificate      /etc/nginx/cert/nifi/server.pem;
ssl_certificate_key  /etc/nginx/cert/nifi/server.key;

      #require clients to authenticate with certs that are part of CA
      #force mutual SSL
      ssl_client_certificate /etc/nginx/cert/nifi/ca.pem;
      ssl_verify_client on;

      ssl on;

	 location / {
                #give the reverse proxy server a client certificate for nifi
                proxy_ssl_certificate /etc/nginx/cert/nifi/rproxy.pem;
                proxy_ssl_certificate_key /etc/nginx/cert/nifi/rproxy.key;
                
                #note that you have to give this client the "proxy user requests"
                #access policy so that it can pass on clients using the
                # X-ProxiedEntitiesChain header -- only valid under mutual ssl

                #do some other proxy optimization
                proxy_buffering off;

                #the following are special headers needed by nifi to build proper urls
                proxy_set_header X-ProxyScheme https;
                proxy_set_header X-ProxyHost $host; 
                proxy_set_header X-ProxyPort $server_port;
                proxy_set_header X-ProxiedEntitiesChain "<${ssl_client_s_dn}>";

                #talk to nifi server bound to localhost
                proxy_pass https://127.0.0.1:8443;
        }
#-- end sniplet

Attempt #2 -
I'm able to configure the nginx to listen on 8443 when bound to the external ip and have the
nifi service bound to 8443 on the localhost/loopback interface- this configuraiton allows
the URLS provided over site to site will work but another issue arrises.  I get a wierd error
on the client that states:

o.a.nifi.remote.StandardRemoteGroupPort RemoteGroupPort[name=nifiout,targets=https://rproxy:8443/nifi]
failed to communicate with https://rproxy:8443/nifi due to org.apache.nifi.remote.exception.HandshakeException:
StandardRootGroupPort[id=<redacted>] authorization failed for user <redacted>
because Unable to modify data transfers to Output Port with ID <redacted>. Contact the
system administrator.

What I have been able to determine is that the site to site data access service is not able
to handle the X-ProxiedEntitiesChain to get access the data api.

Attempt #3 -
I can get site to site working through a reverse proxy by tricking out the nifi systems to
think they are talking directly to each other.  In this configuration I bind the nifi to localhost:8443,
bind the reverse proxy on the same host to externalIP:8443 and replicate the nifi client certificates
on the nifi endpoint to the nginx proxy.  The nifi client connect through the reverse proxy
and the reverse proxy pretends to be the nifi client to talk to the localhost server.  This
works.

This looks like:

      #--- the following is a snipplet from an nginx configuration
	 listen $externalip:8443 ssl default_server;
	 ssl_certificate      /etc/nginx/cert/nifi/server.pem;
      ssl_certificate_key  /etc/nginx/cert/nifi/server.key;

      #require clients to authenticate with certs that are part of CA
      #force mutual SSL
      ssl_client_certificate /etc/nginx/cert/nifi/ca.pem;
      ssl_verify_client on;

      ssl on;

	 location / {
                #give the reverse proxy server a client certificate for nifi
                proxy_ssl_certificate /etc/nginx/cert/nifi/nifi_client.pem;
                proxy_ssl_certificate_key /etc/nginx/cert/nifi/nifi_client.key;
                #this is replicated from the connecting nifi-client
		 
                #do some other proxy optimization
                proxy_buffering off;

                #talk to nifi server bound to localhost
                proxy_pass https://127.0.0.1:8443;
        }
	   #-- end sniplet

I am able to trick out a reverse proxy in order to get site to site working, but would prefer
a configuration option that supports site to site data access through an alternative reverse
proxy hostname and port using either a server or client configuration alternative.  Currently
the site to site proxy configuration is based on the assumption of a client proxy rather than
a server-site reverse proxy.



> incorrect data access urls when using site to site when connecting through reverse proxies.
 Inability for site to site data access urls to utilize proxied user identites.
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: NIFI-4273
>                 URL: https://issues.apache.org/jira/browse/NIFI-4273
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Core Framework
>    Affects Versions: 1.3.0
>         Environment: ubuntu 16.04 64bit, openjdk version "1.8.0_131"
> OpenJDK Runtime Environment (build 1.8.0_131-8u131-b11-2ubuntu1.16.04.3-b11)
> OpenJDK 64-Bit Server VM (build 25.131-b11, mixed mode)
>            Reporter: WaterslideKid
>            Priority: Minor
>
> Issue:
>    Data access using site-to-site binding is based on local environment and does not
easily support a reverse proxy configuration.  The UI works great under a reverse proxy configuration,
but a site to site configuration fails.  It appears that the underlying site to site data
access URLs are built from local environment information rather than configuration and do
not honor the SSL X-ProxiedEntitiesChain or other reverse proxy context information to build
proper URLs.
> My configuration:
> I have two nifi systems that I want to connect to each other via NIFI site to site. 
One is monitoring logs and intending to publish these logs by offering a site to site web
service/server in order to be collected by a client nifi.  My desire is to have the client
nifi contact the server to pull logs by connecting through a nginx reverse proxy.  All of
this is done over SSL using mutual SSL.  I'm able to get these systems talking to each over
directly but having issues talking through a reverse proxy.
> In my preferred configuration, the nifi server behind the reverse proxy is bound to localhost
on port 8443.  The reverse proxy is bound to an external ip address on port 4443.  The client
is set up to talk to the server using site to iste over the reverse proxy on the external
ip on port 4443.  In the UI, the configuration is able to view the flow ports for site to
site but data refuses to flow.  The client complains that it is unable to talk with the data
channel via site to site to the server because it is unable to connect to <hostname>:8443
- which is incorrect.  It appears that the site to site data channel gets its configuration
by choosing the hostname and port of the nifi service rather than obeying the reverse proxy
configuration of the UI.  There is no way to configure the client or server to use an alternative
port or hostname for site to site data.    
>    I have a successful configuration of a mutual SSL enabled NIFI reverse proxy that
allows UI clients to access a NIFI server and pass on client SSL information.
>    Here is my site to site configuration
> {code:java}
> # --- the following is a snipplet from an nginx configuration
> listen 4443 ssl default_server;
> ssl_certificate      /etc/nginx/cert/nifi/server.pem;
> ssl_certificate_key  /etc/nginx/cert/nifi/server.key;
>       #require clients to authenticate with certs that are part of CA
>       #force mutual SSL
>       ssl_client_certificate /etc/nginx/cert/nifi/ca.pem;
>       ssl_verify_client on;
>       ssl on;
> 	 location / {
>                 #give the reverse proxy server a client certificate for nifi
>                 proxy_ssl_certificate /etc/nginx/cert/nifi/rproxy.pem;
>                 proxy_ssl_certificate_key /etc/nginx/cert/nifi/rproxy.key;
>                 
>                 #note that you have to give this client the "proxy user requests"
>                 #access policy so that it can pass on clients using the
>                 # X-ProxiedEntitiesChain header -- only valid under mutual ssl
>                 #do some other proxy optimization
>                 proxy_buffering off;
>                 #the following are special headers needed by nifi to build proper urls
>                 proxy_set_header X-ProxyScheme https;
>                 proxy_set_header X-ProxyHost $host; 
>                 proxy_set_header X-ProxyPort $server_port;
>                 proxy_set_header X-ProxiedEntitiesChain "<${ssl_client_s_dn}>";
>                 #talk to nifi server bound to localhost
>                 proxy_pass https://127.0.0.1:8443;
>         }
> #-- end sniplet
> {code}
> Attempt #2 -
> I'm able to configure the nginx to listen on 8443 when bound to the external ip and have
the nifi service bound to 8443 on the localhost/loopback interface- this configuraiton allows
the URLS provided over site to site will work but another issue arrises.  I get a wierd error
on the client that states:
> o.a.nifi.remote.StandardRemoteGroupPort RemoteGroupPort[name=nifiout,targets=https://rproxy:8443/nifi]
failed to communicate with https://rproxy:8443/nifi due to org.apache.nifi.remote.exception.HandshakeException:
StandardRootGroupPort[id=<redacted>] authorization failed for user <redacted>
because Unable to modify data transfers to Output Port with ID <redacted>. Contact the
system administrator.
> What I have been able to determine is that the site to site data access service is not
able to handle the X-ProxiedEntitiesChain to get access the data api.
> Attempt #3 -
> I can get site to site working through a reverse proxy by tricking out the nifi systems
to think they are talking directly to each other.  In this configuration I bind the nifi to
localhost:8443, bind the reverse proxy on the same host to externalIP:8443 and replicate the
nifi client certificates on the nifi endpoint to the nginx proxy.  The nifi client connect
through the reverse proxy and the reverse proxy pretends to be the nifi client to talk to
the localhost server.  This works.
> This looks like:
>       #--- the following is a snipplet from an nginx configuration
> 	 listen $externalip:8443 ssl default_server;
> 	 ssl_certificate      /etc/nginx/cert/nifi/server.pem;
>       ssl_certificate_key  /etc/nginx/cert/nifi/server.key;
>       #require clients to authenticate with certs that are part of CA
>       #force mutual SSL
>       ssl_client_certificate /etc/nginx/cert/nifi/ca.pem;
>       ssl_verify_client on;
>       ssl on;
> 	 location / {
>                 #give the reverse proxy server a client certificate for nifi
>                 proxy_ssl_certificate /etc/nginx/cert/nifi/nifi_client.pem;
>                 proxy_ssl_certificate_key /etc/nginx/cert/nifi/nifi_client.key;
>                 #this is replicated from the connecting nifi-client
> 		 
>                 #do some other proxy optimization
>                 proxy_buffering off;
>                 #talk to nifi server bound to localhost
>                 proxy_pass https://127.0.0.1:8443;
>         }
> 	   #-- end sniplet
> I am able to trick out a reverse proxy in order to get site to site working, but would
prefer a configuration option that supports site to site data access through an alternative
reverse proxy hostname and port using either a server or client configuration alternative.
 Currently the site to site proxy configuration is based on the assumption of a client proxy
rather than a server-site reverse proxy.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message