nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fullarton, Ben MR" <ben.fullar...@defence.gov.au>
Subject NiFi Site-to-Site error [SEC=UNCLASSIFIED]
Date Thu, 06 Feb 2020 21:22:58 GMT
UNCLASSIFIED

Hi Apache NiFi Team,

Hoping you can help with an error Im getting with our NiFi1.8.0 Site-to-Site running over
a low bandwidth network.

Basically we are testing the performance of NiFi over poor networks . Our 2 NiFi servers are
built on Windows 2016 servers with site-to-site communications enabled and secured with SSL
certificates. We have configured the servers as follows;

Sender (Client) NiFi Server
OS: Windows Server 2016 standard
NiFi Version: 1.8.0
NiFi.Properties:
   # Site to Site properties:
   nifi.remote.input.host=senderhostname
   nifi.remote.input.secure=true
   nifi.remote.input.socket.port=8888
   nifi.remote.input.http.enabled=true
   nifi.remote.input.http.transaction.ttl=600 sec
   nifi.remote.contents.cache.expiration=10 secs
Remote Process Group details:
   URLs: https://Destinationhostname:8443/nifi/
   Transport Protocol: RAW
   Communication Timeout: 60 sec
   Yield Duration: 1 sec

Destination (Server) NiFi Server
OS: Windows Server 2016 standard
NiFi Version: 1.8.0
NiFi.Properties:
   # Site to Site properties:
   nifi.remote.input.host=destinationhostname
   nifi.remote.input.secure=true
   nifi.remote.input.socket.port=8888
   nifi.remote.input.http.enabled=true
   nifi.remote.input.http.transaction.ttl=600 sec
   nifi.remote.contents.cache.expiration=10 secs


So far, sending files between the servers via remote port group is fine under normal conditions.
However when we restrict the network link throughput (channel capacity)  via our network emulator
(iTrinegy INE)  to 100kbps the NiFi client (sender) fails to transfer data and provides the
following error message [sensitive details have been reacted];

RemoteGroupPort[name=senderhostname,targets=https://destinationhostname.fqdn:8443/nifi/] failed
to communicate with remote NiFi instance due to java.io.IOException: Failed to confirm transaction
with Peer[url=nifi://destinationhostname:8888] due to java.net.SocketTimeoutException: Timed
out reading from socket connected to destinationhostname:8888

Consequently, the files stop transmitting to the destination NiFi server.

Can you please advise on how we can fix this error whilst maintaining a network link throughput
(channel capacity) less than or equal to (≤) 100 Kbps? Perhaps we may be able to change
some aspect of the Nifi configuration or underlying configuration files.

If you are able to provide any additional guidance about configuring NiFi to work over low
throughput, high latency and high packet loss links it would be greatly appreciated.

If not able to provide us advice, are you able to suggest someone who may assist us?

Many thanks


Kind Regards,

Ben Fullarton
Systems Engineer
Contractor to Defence
Ph: 02 6265 0158
Mob: 0427 800 745
email: ben.fullarton@defence.gov.au<mailto:ben.fullarton@defence.gov.au>
IMPORTANT: This email remains the property of the Department of Defence. Unauthorised communication
and dealing with the information in the email may be a serious criminal offence. If you have
received this email in error, you are requested to contact the sender and delete the email
immediately.



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message