hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pengcheng Xiong (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-15641) Hive/Druid integration: filter on timestamp not pushed to DruidQuery
Date Sat, 25 Mar 2017 21:48:42 GMT

     [ https://issues.apache.org/jira/browse/HIVE-15641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Pengcheng Xiong updated HIVE-15641:
-----------------------------------
    Target Version/s: 3.0.0  (was: 2.2.0)

> Hive/Druid integration: filter on timestamp not pushed to DruidQuery
> --------------------------------------------------------------------
>
>                 Key: HIVE-15641
>                 URL: https://issues.apache.org/jira/browse/HIVE-15641
>             Project: Hive
>          Issue Type: Improvement
>          Components: Druid integration
>    Affects Versions: 2.2.0
>            Reporter: Jesus Camacho Rodriguez
>            Assignee: Jesus Camacho Rodriguez
>
> It seems we are missing opportunity push Filter operation to DruidQuery.
> For instance, for the following query:
> {code:sql}
> EXPLAIN
> SELECT i_brand_id, floor_day(`__time`), max(ss_quantity), sum(ss_wholesale_cost) as s
> FROM store_sales_sold_time_subset
> WHERE floor_day(`__time`) BETWEEN '1999-11-01 00:00:00' AND '1999-11-10 00:00:00'
> GROUP BY i_brand_id, floor_day(`__time`)
> ORDER BY s
> LIMIT 10;
> OK
> Plan optimized by CBO.
> Vertex dependency in root stage
> Reducer 2 <- Map 1 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> Stage-0
>   Fetch Operator
>     limit:10
>     Stage-1
>       Reducer 3 vectorized
>       File Output Operator [FS_17]
>         Limit [LIM_16] (rows=1 width=0)
>           Number of rows:10
>           Select Operator [SEL_15] (rows=1 width=0)
>             Output:["_col0","_col1","_col2","_col3"]
>           <-Reducer 2 [SIMPLE_EDGE] vectorized
>             SHUFFLE [RS_14]
>               Group By Operator [GBY_13] (rows=1 width=0)
>                 Output:["_col0","_col1","_col2","_col3"],aggregations:["max(VALUE._col0)","sum(VALUE._col1)"],keys:KEY._col0,
KEY._col1
>               <-Map 1 [SIMPLE_EDGE]
>                 SHUFFLE [RS_5]
>                   PartitionCols:_col0, _col1
>                   Group By Operator [GBY_4] (rows=1 width=0)
>                     Output:["_col0","_col1","_col2","_col3"],aggregations:["max(_col2)","sum(_col3)"],keys:_col0,
_col1
>                     Select Operator [SEL_2] (rows=1 width=0)
>                       Output:["_col0","_col1","_col2","_col3"]
>                       Filter Operator [FIL_12] (rows=1 width=0)
>                         predicate:floor_day(__time) BETWEEN '1999-11-01 00:00:00' AND
'1999-11-10 00:00:00'
>                         TableScan [TS_0] (rows=15888 width=0)
>                           tpcds_druid_10@store_sales_sold_time_subset,store_sales_sold_time_subset,Tbl:PARTIAL,Col:NONE,Output:["__time","i_brand_id","ss_quantity","ss_wholesale_cost"],properties:{"druid.query.json":"{\"queryType\":\"select\",\"dataSource\":\"druid_tpcds_ss_sold_time_subset\",\"descending\":false,\"intervals\":[\"1900-01-01T00:00:00.000Z/3000-01-01T00:00:00.000Z\"],\"dimensions\":[\"i_item_id\",\"i_rec_start_date\",\"i_rec_end_date\",\"i_item_desc\",\"i_brand_id\",\"i_brand\",\"i_class_id\",\"i_class\",\"i_category_id\",\"i_category\",\"i_manufact_id\",\"i_manufact\",\"i_size\",\"i_formulation\",\"i_color\",\"i_units\",\"i_container\",\"i_manager_id\",\"i_product_name\",\"c_customer_id\",\"c_salutation\",\"c_first_name\",\"c_last_name\",\"c_preferred_cust_flag\",\"c_birth_day\",\"c_birth_month\",\"c_birth_year\",\"c_birth_country\",\"c_login\",\"c_email_address\",\"c_last_review_date\",\"ca_address_id\",\"ca_street_number\",\"ca_street_name\",\"ca_street_type\",\"ca_suite_number\",\"ca_city\",\"ca_county\",\"ca_state\",\"ca_zip\",\"ca_country\",\"ca_gmt_offset\",\"ca_location_type\",\"s_store_id\",\"s_rec_start_date\",\"s_rec_end_date\",\"s_store_name\",\"s_hours\",\"s_manager\",\"s_market_id\",\"s_geography_class\",\"s_market_desc\",\"s_market_manager\",\"s_division_id\",\"s_division_name\",\"s_company_id\",\"s_company_name\",\"s_street_number\",\"s_street_name\",\"s_street_type\",\"s_suite_number\",\"s_city\",\"s_county\",\"s_state\",\"s_zip\",\"s_country\",\"s_gmt_offset\"],\"metrics\":[\"ss_ticket_number\",\"ss_quantity\",\"ss_wholesale_cost\",\"ss_list_price\",\"ss_sales_price\",\"ss_ext_discount_amt\",\"ss_ext_sales_price\",\"ss_ext_wholesale_cost\",\"ss_ext_list_price\",\"ss_ext_tax\",\"ss_coupon_amt\",\"ss_net_paid\",\"ss_net_paid_inc_tax\",\"ss_net_profit\",\"i_current_price\",\"i_wholesale_cost\",\"s_number_employees\",\"s_floor_space\",\"s_tax_precentage\"],\"granularity\":\"all\",\"pagingSpec\":{\"threshold\":16384},\"context\":{\"druid.query.fetch\":false}}","druid.query.type":"select"}
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message