spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ruslan Dautkhanov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-17076) Cardinality estimation of join operator
Date Tue, 15 Nov 2016 01:18:58 GMT

    [ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665644#comment-15665644
] 

Ruslan Dautkhanov commented on SPARK-17076:
-------------------------------------------

HIVE-13076 added FK constraint to Hive / HMS. 
Joining two tables without declared relation can produce and number of rows.
On one extreme case, joining two tables produces zero rows (if there is no overlaps in those
join set of ids).
On another extream case, joining two tables will produce N x M rows where N and M are number
of rows in each case.
Luckliy, most often joining happens using columns that can have declared constraints on them.
For example,
 customers.customer_id = purchases.customer_id AND purchases.product_id = products.product_id
If you'd build on HIVE-13076, Spark CBO will know exactly that  customers.customer_id = purchases.customer_id
AND purchases.product_id = products.product_id will always produce 1 row for each join (join
cardinality = # of rows on the biggest table).

> Cardinality estimation of join operator
> ---------------------------------------
>
>                 Key: SPARK-17076
>                 URL: https://issues.apache.org/jira/browse/SPARK-17076
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Optimizer
>    Affects Versions: 2.0.0
>            Reporter: Ron Hu
>
> support cardinality estimates for equi-join, Cartesian product join, and outer join,
etc. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message