hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sahil Takiar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-18652) Print Spark metrics on console
Date Thu, 08 Feb 2018 02:03:00 GMT
Sahil Takiar created HIVE-18652:
-----------------------------------

             Summary: Print Spark metrics on console
                 Key: HIVE-18652
                 URL: https://issues.apache.org/jira/browse/HIVE-18652
             Project: Hive
          Issue Type: Sub-task
          Components: Spark
            Reporter: Sahil Takiar


For Hive-on-MR, each MR job launched prints out some stats about the job:

{code}
INFO  : 2018-02-07 17:51:11,218 Stage-1 map = 0%,  reduce = 0%
INFO  : 2018-02-07 17:51:18,396 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.87 sec
INFO  : 2018-02-07 17:51:25,742 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.34 sec
INFO  : MapReduce Total cumulative CPU time: 4 seconds 340 msec
INFO  : Ended Job = job_1517865654989_0004
INFO  : MapReduce Jobs Launched:
INFO  : Stage-Stage-1: Map: 1  Reduce: 1   Cumulative CPU: 4.34 sec   HDFS Read: 7353 HDFS
Write: 151 SUCCESS
INFO  : Total MapReduce CPU Time Spent: 4 seconds 340 msec
{code}

We should do the same for Hive-on-Spark.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message