storm-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [storm] Ethanlm opened a new pull request #3313: [STORM-3679] Fix the misuse of nodeId as hostname in LoadAwareShuffleGrouping
Date Tue, 21 Jul 2020 19:28:30 GMT

Ethanlm opened a new pull request #3313:
URL: https://github.com/apache/storm/pull/3313


   ## What is the purpose of the change
   
   LoadAwareShuffleGrouping misuses nodeId as the hostname to cause consistent invalid DNS
queries for "hostnames" like 2a1f2cf3-c701-4621-9e93-640b4e63be48-10.215.73.209.
   
   This causes excessive unnecessary loads on nscd and DNS. Also because of this bug, every
target tasks will be treated as at least RACK_LOCAL because if an ip address can't be determined,
YahooDNSToSwitchMapping treats it as DEFAULT_RACK. This doesn't impact WORKER_LOCAL and HOST_LOCAL
though.
   
   ## How was the change tested
   Before the change,
   
   added logs and showed that the "hostname" is actually an ID. And tcp dump
   ```
   18:42:21.913688 IP <host1>.42462 > <dns>: 43104+ A? 2a1f2cf3-c701-4621-9e93-640b4e63be48-<ip1>.
(68)
   18:42:21.914181 IP <dns> > <host1>.42462: 43104 NXDomain 0/1/0 (143)
   ```
   
   After the change, 
   determined from additional added debug logs (removed after the coding is done) that the
hostnames are correct and no invalid queries shown in TCP dump.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message