drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@dremio.com>
Subject Calcite: Trait propagation using relset iteration versus remove extraneous trait creation
Date Mon, 14 Mar 2016 22:09:48 GMT
Hey All,

I've been thinking about the SubsetTransformer pattern [1] that we use in
Drill to ensure trait propagation. It was discussed here in Calcite [2]

Julian's felt that the correct solution (and the patch he ultimately
applied) was to use a create and then remove behavior. Take a look at his
revision to my test here [3] where he adds the SortRemoveRule in order to
remove an extraneous Sort operation.

It seems like we need to either introduce a new mechanism in Calcite to
accomplish this or we need to adopt the removal behavior. (I also believe
there are a small set of situations where we insert distribution for
parallelization purposes as opposed to a requirement for a particular
operation... we'll need to determine how those work and figure out how to
express correctly in this removal pattern.)

Thoughts?

[1]
https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/SubsetTransformer.java
[2] https://issues.apache.org/jira/browse/CALCITE-606
[3]
https://github.com/julianhyde/calcite/commit/fb203dc4b9aea89bfed839c22ae3e285044df400#diff-9494b27dde1061ef95e3853cb6222b5bR103
--
Jacques Nadeau
CTO and Co-Founder, Dremio

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message