storm-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From roshannaik <...@git.apache.org>
Subject [GitHub] storm issue #2241: STORM-2306 : Messaging subsystem redesign.
Date Mon, 06 Nov 2017 23:30:22 GMT
Github user roshannaik commented on the issue:

    https://github.com/apache/storm/pull/2241
  
    @revans2 thanks very much for the offer to help, i think it might be useful to get past
this issue that is blocking this. 
    
    Just updated the PR with these two key changes. 
    
    - Discovered that workers need to communicate their internal back pressure situation to
other workers so that they stop/resume sending messages to components experiencing BP.  
    - Now Bolts also use non-blocking emits so that in case of BP they can continue to process
metrics ticks. 
    
    
    **Curent Status:**
    - Some new metrics need for monitoring overflow needs to be introduced. Also some minor
todos mentioned in code need to be addressed.
    - During scale testing, found that under certain multi-worker scenarios, noticed a inexplicable
performance drop. The simplest way to reproduce the issue appears to be using a 2 workers
setup with spouts and bolts distributed so that there is messaging occurring in both directions.
Don't have a good grip on the issue yet.  Unfortunately last 3 weeks had to switch my attention
to some other pressing release centric tasks at work.
    
    May be useful to sync up with you offline soon and see if looking into the issue together
helps.


---

Mime
View raw message