Mailing-List: contact reviews-help@spark.apache.org; run by ezmlm
Precedence: bulk
From: vanzin <git@git.apache.org>
To: reviews@spark.apache.org
Reply-To: reviews@spark.apache.org
References: <git-pr-18004-spark@git.apache.org>
In-Reply-To: <git-pr-18004-spark@git.apache.org>
Subject: [GitHub] spark issue #18004: [SPARK-18838][CORE] Introduce blocking strategy for Live...
Content-Type: text/plain
Message-Id: <20170606164657.1ADA7DFE61@git1-us-west.apache.org>
Date: Tue,  6 Jun 2017 16:46:57 +0000 (UTC)
archived-at: Tue, 06 Jun 2017 16:46:59 -0000

Github user vanzin commented on the issue:

    https://github.com/apache/spark/pull/18004
  
    > But the key one is that it will change the synchronization paradigm
    
    If you read the discussion in the other PR, we pushed back and the current design avoids exactly those issues. That's why there isn't a queue for each listener, but multiple listener share a single queue.
    
    > The asynchronous mechanism will be implemented in a very different way for all listeners.
    
    You're talking policy, not mechanism. Asynchronous means asynchronous; there's a producer and a consumer. If you think that different consumers need different policies for how to consume/block/discard data, you can implement that.
    
    > The key thing in the event logging listener is the ability to not queue the blockUpdated messages
    
    That's such a small gain that I'm not sure why it's such a big deal... in any case, see above, policy vs. mechanism. Your current approach pushes *all* the burden of handling things asynchronously to the listeners. That's a huge source of duplicated code if you plan to do this for multiple listeners.
    
    > For the couple storageStatusListener / storageListener
    
    I'd just like to point out that in a separate bug I'm basically deleting all these UI listeners and replacing them with a new one, so if you're spending too much time looking at them, you'll probably be wasting a lot of that effort.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org