spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From gengliangwang <...@git.apache.org>
Subject [GitHub] spark pull request #20955: [SPARK-23838][WEBUI] Running SQL query is display...
Date Sun, 01 Apr 2018 07:17:11 GMT
GitHub user gengliangwang opened a pull request:

    https://github.com/apache/spark/pull/20955

    [SPARK-23838][WEBUI] Running SQL query is displayed as "completed" in SQL tab

    ## What changes were proposed in this pull request?
    
    A running SQL query would appear as completed in the Spark UI:
    ![image1](https://user-images.githubusercontent.com/1097932/38170733-3d7cb00c-35bf-11e8-994c-43f2d4fa285d.png)
    
    We can see the query in "Completed queries", while in in the job page we see it's still
running Job 132.
    ![image2](https://user-images.githubusercontent.com/1097932/38170735-48f2c714-35bf-11e8-8a41-6fae23543c46.png)
    
    After some time in the query still appears in "Completed queries" (while it's still running),
but the "Duration" gets increased.
    ![image3](https://user-images.githubusercontent.com/1097932/38170737-50f87ea4-35bf-11e8-8b60-000f6f918964.png)
    
    To reproduce, we can run a query with multiple jobs. E.g. Run TPCDS q6.
    
    The reason is that updates from executions are written into kvstore periodically, and
the job start event may be missing.
    
    ## How was this patch tested?
    Manually run the job again and check the SQL Tab. The fix is pretty simple.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/gengliangwang/spark jobCompleted

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/20955.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #20955
    
----
commit b418feb6ac4bb7523c50ea27f74d3b02c310956a
Author: Wang Gengliang <gengliang.wang@...>
Date:   2018-04-01T07:08:06Z

    Fix sql tab

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message