hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-4318) OperatorHooks hit performance even when not used
Date Tue, 09 Apr 2013 07:38:15 GMT
Gopal V created HIVE-4318:
-----------------------------

             Summary: OperatorHooks hit performance even when not used
                 Key: HIVE-4318
                 URL: https://issues.apache.org/jira/browse/HIVE-4318
             Project: Hive
          Issue Type: Bug
          Components: Query Processor
         Environment: Ubuntu LXC (64 bit)
            Reporter: Gopal V


Operator Hooks inserted into Operator.java cause a performance hit even when it is not being
used.

For a count(1) query tested with & without the operator hook calls.

{code:title=with}
2013-04-09 07:33:58,920 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 84.07 sec
Total MapReduce CPU Time Spent: 1 minutes 24 seconds 70 msec
OK
28800991
Time taken: 40.407 seconds, Fetched: 1 row(s)
{code}

{code:title=without}
2013-04-09 07:33:02,355 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 68.48 sec
...
Total MapReduce CPU Time Spent: 1 minutes 8 seconds 480 msec
OK
28800991
Time taken: 35.907 seconds, Fetched: 1 row(s)
{code}

The effect is multiplied by the number of operators in the pipeline that has to forward the
row - the more operators there are the, the slower the query.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message