beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anton Kedin (JIRA)" <>
Subject [jira] [Commented] (BEAM-3417) Fix Calcite assertions
Date Thu, 08 Mar 2018 04:30:00 GMT


Anton Kedin commented on BEAM-3417:

*What fails?

[Assert in question is in in VolcanoPlanner |].
It checks whether [all traits are simple|]
by checking whether they're not instances of RelCompositeTrait.

*Why it fails?

In our case, when it fails, traitSet.allSimple() has 2 traits. One is BeamLogicalConvention
(it's not a composite trait), and another is a collation-related composite trait which causes
the assertion to fail. 

*Where does the composite trait come from?

We specify the collation trait def in [BeamQueryPlanner|]
before parsing. It then [gets replaced in LogicalTableScan|]
with the [composite trait|]
which causes the failure.

*Why LogicalTableScan needs to do the collation magic?

Dunno, it seems that it adds the statistics information to the collation trait so that the
engine can handle sorting correctly. It does so only when we ask it to by adding the collation
trait def.

*Why VolcanoPlanner doesn't like CompositeTraitSet in that part?


*Do we need the collation trait def?


*What do we do?

If we can, it probably makes sense to replace LogicalTableScanRel with some kind of BeamIllogicalPCollectionScan
which doesn't do all the collation magic or makes it configurable

> Fix Calcite assertions
> ----------------------
>                 Key: BEAM-3417
>                 URL:
>             Project: Beam
>          Issue Type: Task
>          Components: dsl-sql
>            Reporter: Anton Kedin
>            Priority: Major
> Currently we disable assertions in test for every project which depends on Beam SQL /
Calcite. Otherwise it fails assertions when Calcite validates relational representation of
the query. E.g. in projects which depend on Beam SQL / Calcite we have to specify 
> {code:java|title=build.gradle}
> test {
>  jvmArgs "-da" 
> }
> {code}
> We need to either update our relational conversion logic or come up with some other solution
so that we don't have to disable assertions globally. If it's an incorrect assertion in Calcite
then we need to fix it there.

This message was sent by Atlassian JIRA

View raw message