spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vanzin <...@git.apache.org>
Subject [GitHub] spark issue #18004: [SPARK-18838][CORE] Introduce blocking strategy for Live...
Date Tue, 06 Jun 2017 16:46:57 GMT
Github user vanzin commented on the issue:

    https://github.com/apache/spark/pull/18004
  
    > But the key one is that it will change the synchronization paradigm
    
    If you read the discussion in the other PR, we pushed back and the current design avoids
exactly those issues. That's why there isn't a queue for each listener, but multiple listener
share a single queue.
    
    > The asynchronous mechanism will be implemented in a very different way for all listeners.
    
    You're talking policy, not mechanism. Asynchronous means asynchronous; there's a producer
and a consumer. If you think that different consumers need different policies for how to consume/block/discard
data, you can implement that.
    
    > The key thing in the event logging listener is the ability to not queue the blockUpdated
messages
    
    That's such a small gain that I'm not sure why it's such a big deal... in any case, see
above, policy vs. mechanism. Your current approach pushes *all* the burden of handling things
asynchronously to the listeners. That's a huge source of duplicated code if you plan to do
this for multiple listeners.
    
    > For the couple storageStatusListener / storageListener
    
    I'd just like to point out that in a separate bug I'm basically deleting all these UI
listeners and replacing them with a new one, so if you're spending too much time looking at
them, you'll probably be wasting a lot of that effort.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message