spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [spark] shahidki31 edited a comment on issue #26378: [SPARK-29724][SPARK-29726][WEBUI][SQL] Support JDBC/ODBC tab for HistoryServer WebUI
Date Tue, 26 Nov 2019 23:07:22 GMT
shahidki31 edited a comment on issue #26378: [SPARK-29724][SPARK-29726][WEBUI][SQL] Support
JDBC/ODBC tab for HistoryServer WebUI
URL: https://github.com/apache/spark/pull/26378#issuecomment-558852948
 
 
   > LGTM on the rest, but would be helpful if another set of eyes could look on the store
updating logic, because I don't fully know how the flushing, making sure that updates are
synced etc. works.
   
   Thanks @juliuszsompolski . I have slightly refactored the store update logic 
   
   Ideally in all the listeners, the store logic goes like this,
   **LiveUI:** update the store whenever an event comes. To prevent frequent updates, if there
are frequent events like tasks event, there is a threshold `liveUpdatePeriod`, so that after
the threshold + last written time we update the store.
   
   **HistoryUI:** Keep all the events data in a hash map and update the store, after the last
event occurs,(or during flush). Here, I am also updating when the session closed event, and
execution closed event, so that we can remove the map entries. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message