hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-9382) Query got rerun with Global Limit optimization on and Fetch optimization off
Date Thu, 15 Jan 2015 01:16:34 GMT

     [ https://issues.apache.org/jira/browse/HIVE-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Wei Zheng updated HIVE-9382:
----------------------------
    Description: 
When Global Limit optimization is enabled, and Fetch Optimization for Simple Queries is off
or not applicable, some queries with LIMIT clause will run twice.
set hive.limit.optimize.enable=true;
set hive.fetch.task.conversion=none;

For example,
{code:sql}
hive> select * from t1 limit 10;
Query ID = wzheng_20150107185252_4a6d0e65-9e58-464b-9ed3-9177740c30a9
Total jobs = 1
Launching Job 1 out of 1


Status: Running (Executing on YARN cluster with App id application_1420567249453_0039)

--------------------------------------------------------------------------------
        VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
--------------------------------------------------------------------------------
Map 1 ..........   SUCCEEDED      1          1        0        0       0       0
--------------------------------------------------------------------------------
VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 0.41 s
--------------------------------------------------------------------------------
OK
201208	99848	119820	32627	982976	509206	0.000100898
201208	99745	119820	32627	982976	509206	0.000100898
201208	99739	119820	32627	982976	509206	0.000100898
201208	99847	119820	32627	982976	509206	0.000100898
201208	613588	119820	32627	982976	509206	0.000100898
201208	99809	119820	32627	982976	509206	0.000100898
201208	99725	119820	32627	982976	509206	0.000100898
201208	99666	119820	32627	982976	509206	0.000100898
201208	99743	119820	32627	982976	509206	0.000100898
201208	99801	119820	32627	982976	509206	0.000100898
Retry query with a different approach...
Query ID = wzheng_20150107185252_8a77f793-cad7-4c6b-b64a-07d8310970b9
Total jobs = 1
Launching Job 1 out of 1


Status: Running (Executing on YARN cluster with App id application_1420567249453_0039)

--------------------------------------------------------------------------------
        VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
--------------------------------------------------------------------------------
Map 1 ..........   SUCCEEDED    309        309        0        0       0       0
--------------------------------------------------------------------------------
VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 2.04 s
--------------------------------------------------------------------------------
OK
201208	99848	119820	32627	982976	509206	0.000100898
201208	99745	119820	32627	982976	509206	0.000100898
201208	99739	119820	32627	982976	509206	0.000100898
201208	99847	119820	32627	982976	509206	0.000100898
201208	613588	119820	32627	982976	509206	0.000100898
201208	99809	119820	32627	982976	509206	0.000100898
201208	99725	119820	32627	982976	509206	0.000100898
201208	99666	119820	32627	982976	509206	0.000100898
201208	99743	119820	32627	982976	509206	0.000100898
201208	99801	119820	32627	982976	509206	0.000100898
Time taken: 2.748 seconds, Fetched: 10 row(s)
{code}

  was:
When Global Limit optimization is enabled, and Fetch Optimization for Simple Queries is off
or not applicable, some queries with LIMIT clause will run twice.
set hive.limit.optimize.enable=true;
set hive.fetch.task.conversion=none;

For example,
{panel}
hive> select * from t1 limit 10;
Query ID = wzheng_20150107185252_4a6d0e65-9e58-464b-9ed3-9177740c30a9
Total jobs = 1
Launching Job 1 out of 1


Status: Running (Executing on YARN cluster with App id application_1420567249453_0039)

--------------------------------------------------------------------------------
        VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
--------------------------------------------------------------------------------
Map 1 ..........   SUCCEEDED      1          1        0        0       0       0
--------------------------------------------------------------------------------
VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 0.41 s
--------------------------------------------------------------------------------
OK
201208	99848	119820	32627	982976	509206	0.000100898
201208	99745	119820	32627	982976	509206	0.000100898
201208	99739	119820	32627	982976	509206	0.000100898
201208	99847	119820	32627	982976	509206	0.000100898
201208	613588	119820	32627	982976	509206	0.000100898
201208	99809	119820	32627	982976	509206	0.000100898
201208	99725	119820	32627	982976	509206	0.000100898
201208	99666	119820	32627	982976	509206	0.000100898
201208	99743	119820	32627	982976	509206	0.000100898
201208	99801	119820	32627	982976	509206	0.000100898
Retry query with a different approach...
Query ID = wzheng_20150107185252_8a77f793-cad7-4c6b-b64a-07d8310970b9
Total jobs = 1
Launching Job 1 out of 1


Status: Running (Executing on YARN cluster with App id application_1420567249453_0039)

--------------------------------------------------------------------------------
        VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
--------------------------------------------------------------------------------
Map 1 ..........   SUCCEEDED    309        309        0        0       0       0
--------------------------------------------------------------------------------
VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 2.04 s
--------------------------------------------------------------------------------
OK
201208	99848	119820	32627	982976	509206	0.000100898
201208	99745	119820	32627	982976	509206	0.000100898
201208	99739	119820	32627	982976	509206	0.000100898
201208	99847	119820	32627	982976	509206	0.000100898
201208	613588	119820	32627	982976	509206	0.000100898
201208	99809	119820	32627	982976	509206	0.000100898
201208	99725	119820	32627	982976	509206	0.000100898
201208	99666	119820	32627	982976	509206	0.000100898
201208	99743	119820	32627	982976	509206	0.000100898
201208	99801	119820	32627	982976	509206	0.000100898
Time taken: 2.748 seconds, Fetched: 10 row(s)
{panel}


> Query got rerun with Global Limit optimization on and Fetch optimization off
> ----------------------------------------------------------------------------
>
>                 Key: HIVE-9382
>                 URL: https://issues.apache.org/jira/browse/HIVE-9382
>             Project: Hive
>          Issue Type: Bug
>          Components: Physical Optimizer
>    Affects Versions: 0.14.0
>            Reporter: Wei Zheng
>            Assignee: Wei Zheng
>
> When Global Limit optimization is enabled, and Fetch Optimization for Simple Queries
is off or not applicable, some queries with LIMIT clause will run twice.
> set hive.limit.optimize.enable=true;
> set hive.fetch.task.conversion=none;
> For example,
> {code:sql}
> hive> select * from t1 limit 10;
> Query ID = wzheng_20150107185252_4a6d0e65-9e58-464b-9ed3-9177740c30a9
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (Executing on YARN cluster with App id application_1420567249453_0039)
> --------------------------------------------------------------------------------
>         VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
> --------------------------------------------------------------------------------
> Map 1 ..........   SUCCEEDED      1          1        0        0       0       0
> --------------------------------------------------------------------------------
> VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 0.41 s
> --------------------------------------------------------------------------------
> OK
> 201208	99848	119820	32627	982976	509206	0.000100898
> 201208	99745	119820	32627	982976	509206	0.000100898
> 201208	99739	119820	32627	982976	509206	0.000100898
> 201208	99847	119820	32627	982976	509206	0.000100898
> 201208	613588	119820	32627	982976	509206	0.000100898
> 201208	99809	119820	32627	982976	509206	0.000100898
> 201208	99725	119820	32627	982976	509206	0.000100898
> 201208	99666	119820	32627	982976	509206	0.000100898
> 201208	99743	119820	32627	982976	509206	0.000100898
> 201208	99801	119820	32627	982976	509206	0.000100898
> Retry query with a different approach...
> Query ID = wzheng_20150107185252_8a77f793-cad7-4c6b-b64a-07d8310970b9
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (Executing on YARN cluster with App id application_1420567249453_0039)
> --------------------------------------------------------------------------------
>         VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
> --------------------------------------------------------------------------------
> Map 1 ..........   SUCCEEDED    309        309        0        0       0       0
> --------------------------------------------------------------------------------
> VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 2.04 s
> --------------------------------------------------------------------------------
> OK
> 201208	99848	119820	32627	982976	509206	0.000100898
> 201208	99745	119820	32627	982976	509206	0.000100898
> 201208	99739	119820	32627	982976	509206	0.000100898
> 201208	99847	119820	32627	982976	509206	0.000100898
> 201208	613588	119820	32627	982976	509206	0.000100898
> 201208	99809	119820	32627	982976	509206	0.000100898
> 201208	99725	119820	32627	982976	509206	0.000100898
> 201208	99666	119820	32627	982976	509206	0.000100898
> 201208	99743	119820	32627	982976	509206	0.000100898
> 201208	99801	119820	32627	982976	509206	0.000100898
> Time taken: 2.748 seconds, Fetched: 10 row(s)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message