hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mostafa Mokhtar (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-10350) CBO: With hive.cbo.costmodel.extended enabled IO cost is negative
Date Wed, 15 Apr 2015 22:59:58 GMT

     [ https://issues.apache.org/jira/browse/HIVE-10350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mostafa Mokhtar updated HIVE-10350:
-----------------------------------
    Attachment:     (was: HIVE-10331.01.patch)

> CBO: With hive.cbo.costmodel.extended enabled IO cost is negative
> -----------------------------------------------------------------
>
>                 Key: HIVE-10350
>                 URL: https://issues.apache.org/jira/browse/HIVE-10350
>             Project: Hive
>          Issue Type: Sub-task
>          Components: CBO
>    Affects Versions: 1.2.0
>            Reporter: Mostafa Mokhtar
>            Assignee: Mostafa Mokhtar
>             Fix For: 1.2.0
>
>         Attachments: HIVE-10331.01.patch
>
>
> Not an overflow but parallelism ends up being -1 as it uses number of buckets
> {code}
>  final int parallelism = RelMetadataQuery.splitCount(join) == null
>               ? 1 : RelMetadataQuery.splitCount(join);
> {code}
> {code}
> 2015-04-13 18:19:09,154 DEBUG [main]: cost.HiveCostModel (HiveCostModel.java:getJoinCost(62))
- COMMON_JOIN cost: {1600892.857142857 rows, 2.4463782008994658E7 cpu, 8.54445445875E10 io}
> 2015-04-13 18:19:09,155 DEBUG [main]: cost.HiveCostModel (HiveCostModel.java:getJoinCost(62))
- MAP_JOIN cost: {1600892.857142857 rows, 1601785.714285714 cpu, -1698787.4999999998 io}
> 2015-04-13 18:19:09,155 DEBUG [main]: cost.HiveCostModel (HiveCostModel.java:getJoinCost(72))
- MAP_JOIN selected
> 2015-04-13 18:19:09,157 DEBUG [main]: parse.CalcitePlanner (CalcitePlanner.java:apply(862))
- Plan After Join Reordering:
> HiveSort(fetch=[100]): rowcount = 6006.726049749041, cumulative cost = {1.1468867492063493E8
rows, 1.166177684126984E8 cpu, -1.1757664816220238E9 io}, id = 3000
>   HiveSort(sort0=[$0], dir0=[ASC]): rowcount = 6006.726049749041, cumulative cost = {1.1468867492063493E8
rows, 1.166177684126984E8 cpu, -1.1757664816220238E9 io}, id = 2998
>     HiveProject(customer_id=[$4], customername=[concat($9, ', ', $8)]): rowcount = 6006.726049749041,
cumulative cost = {1.1468867492063493E8 rows, 1.166177684126984E8 cpu, -1.1757664816220238E9
io}, id = 3136
>       HiveJoin(condition=[=($1, $5)], joinType=[inner], joinAlgorithm=[map_join], cost=[{5.557820341269841E7
rows, 5.557840182539682E7 cpu, -4299694.122023809 io}]): rowcount = 6006.726049749041, cumulative
cost = {1.1468867492063493E8 rows, 1.166177684126984E8 cpu, -1.1757664816220238E9 io}, id
= 3132
>         HiveJoin(condition=[=($0, $1)], joinType=[inner], joinAlgorithm=[map_join], cost=[{5.7498805E7
rows, 5.9419605E7 cpu, -1.15248E9 io}]): rowcount = 5.5578005E7, cumulative cost = {5.7498805E7
rows, 5.9419605E7 cpu, -1.15248E9 io}, id = 3100
>           HiveProject(sr_cdemo_sk=[$4]): rowcount = 5.5578005E7, cumulative cost = {0.0
rows, 0.0 cpu, 0.0 io}, id = 2992
>             HiveTableScan(table=[[tpcds_bin_orc_200.store_returns]]): rowcount = 5.5578005E7,
cumulative cost = {0}, id = 2878
>           HiveProject(cd_demo_sk=[$0]): rowcount = 1920800.0, cumulative cost = {0.0
rows, 0.0 cpu, 0.0 io}, id = 2978
>             HiveTableScan(table=[[tpcds_bin_orc_200.customer_demographics]]): rowcount
= 1920800.0, cumulative cost = {0}, id = 2868
>         HiveJoin(condition=[=($10, $1)], joinType=[inner], joinAlgorithm=[map_join],
cost=[{1787.9365079365077 rows, 1790.15873015873 cpu, -8000.0 io}]): rowcount = 198.4126984126984,
cumulative cost = {1611666.507936508 rows, 1619761.5873015872 cpu, -1.89867875E7 io}, id =
3130
>           HiveJoin(condition=[=($0, $4)], joinType=[inner], joinAlgorithm=[map_join],
cost=[{8985.714285714286 rows, 16185.714285714286 cpu, -1.728E7 io}]): rowcount = 1785.7142857142856,
cumulative cost = {1609878.5714285714 rows, 1617971.4285714284 cpu, -1.89787875E7 io}, id
= 3128
>             HiveProject(hd_demo_sk=[$0], hd_income_band_sk=[$1]): rowcount = 7200.0,
cumulative cost = {0.0 rows, 0.0 cpu, 0.0 io}, id = 2982
>               HiveTableScan(table=[[tpcds_bin_orc_200.household_demographics]]): rowcount
= 7200.0, cumulative cost = {0}, id = 2871
>             HiveJoin(condition=[=($3, $6)], joinType=[inner], joinAlgorithm=[map_join],
cost=[{1600892.857142857 rows, 1601785.714285714 cpu, -1698787.4999999998 io}]): rowcount
= 1785.7142857142856, cumulative cost = {1600892.857142857 rows, 1601785.714285714 cpu, -1698787.4999999998
io}, id = 3105
>               HiveProject(c_customer_id=[$1], c_current_cdemo_sk=[$2], c_current_hdemo_sk=[$3],
c_current_addr_sk=[$4], c_first_name=[$8], c_last_name=[$9]): rowcount = 1600000.0, cumulative
cost = {0.0 rows, 0.0 cpu, 0.0 io}, id = 2970
>                 HiveTableScan(table=[[tpcds_bin_orc_200.customer]]): rowcount = 1600000.0,
cumulative cost = {0}, id = 2862
>               HiveProject(ca_address_sk=[$0], ca_city=[$6]): rowcount = 892.8571428571428,
cumulative cost = {0.0 rows, 0.0 cpu, 0.0 io}, id = 2974
>                 HiveFilter(condition=[=($6, 'Hopewell')]): rowcount = 892.8571428571428,
cumulative cost = {0.0 rows, 0.0 cpu, 0.0 io}, id = 2972
>                   HiveTableScan(table=[[tpcds_bin_orc_200.customer_address]]): rowcount
= 800000.0, cumulative cost = {0}, id = 2864
>           HiveProject(ib_income_band_sk=[$0], ib_lower_bound=[$1], ib_upper_bound=[$2]):
rowcount = 2.2222222222222223, cumulative cost = {0.0 rows, 0.0 cpu, 0.0 io}, id = 2988
>             HiveFilter(condition=[AND(>=($1, 32287), <=($2, +(32287, 50000)))]):
rowcount = 2.2222222222222223, cumulative cost = {0.0 rows, 0.0 cpu, 0.0 io}, id = 2986
>               HiveTableScan(table=[[tpcds_bin_orc_200.income_band]]): rowcount = 20.0,
cumulative cost = {0}, id = 2874
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message