calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Hyde <jh...@apache.org>
Subject Dealing with metadata in cyclic RelNode graphs
Date Mon, 14 Dec 2015 01:13:26 GMT
tl;dr: Please can at least one person from Drill, Hive, Phoenix, and
any other interested stakeholders, review and join the discussion in
https://issues.apache.org/jira/browse/CALCITE-794.

---

We currently have to treat carefully when evaluating metadata (e.g.
row count, whether columns are keys) because RelNode graphs are
allowed to be cyclic. This occurs when you are using the Volcano
planner and it is easy for a sequence of merges to cause subsets to be
ancestors of themselves.

My proposed solution is to require a call context when asking for
metadata. RelMetadataQuery, previously abstract, becomes that context.
It maintains a map of all active metadata calls, and can therefore
detect cycles.

The issue is https://issues.apache.org/jira/browse/CALCITE-794 and
attached to it is a development branch that is ready for review. (The
test suite passes, but a few important tests are disabled, so it's not
ready to commit just yet.)

Please review.

This is an important change, and changes some key APIs, so I would
like a +1 from Calcite stakeholders, including from a least one
committer from each of the Drill, Hive, Phoenix projects, before I
proceed.

I'd like to discuss (a) whether this is the right change, (b) whether
this should go in the 1.6 release (some time this month), (c) whether
this justifies upping Calcite's version to 2.0.

In my opinion, this can go into 1.6 this month, if we can get the
disabled tests to pass and we prove that performance is at par.

Julian

Mime
View raw message