drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacq...@dremio.com>
Subject Re: Drill Test Framework : Planning Tests
Date Tue, 27 Oct 2015 16:25:02 GMT
We should use the structured response (and do structural comparisons) for
plan verification. The text plan should be cleaned up so that it is more
useful and user friendly. We definitely shouldn't depend on what is
effectively a random toString method for functional verification (I know we
do now). We should figure out a way to implement the abstract functionality
verification using a builder pattern or similar. Think Calcite's RelBuilder
with a more generic interfaces. Maybe we create a JsonRelMatcher tree and
then we can use that to verify a plan.

As I started to write up an example expression proposal, I start to think:
this is arbitrary json tree matching, any way we can use something that
already exists. For example, there are probably 100 json libraries that do
basically this...

thoughts?

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Mon, Oct 26, 2015 at 6:32 PM, rahul challapalli <
challapallirahul@gmail.com> wrote:

> Drillers,
>
> We have a handful of tests in the drill test framework (Refer to [1] for
> some examples) which compare the text version of the plan to an expected
> result. The goal is to compare the structure of the text plan and also
> selected attributes of some operators ( Eg : numFiles, list of scanned
> files). The current implementation uses a regex based comparison where the
> QA person constructs an equivalent regex based expected file. This reduces
> the QA productivity and is not flexible w.r.t to moving parts in the plan.
> Any thoughts around this?
>
> Few suggestions :
> 1. Run explain plan with a flag so that the text plan returned is
> deterministic and does not contain any moving parts. This version of the
> plan returned can contain the list of files scanned in an alphabetically
> sorted order or in some way which is deterministic.
> 2. Use the JSON plan and do an operator DAG based comparison. This can be
> very flexible. Even with this we still need to have some tests against the
> Text version of the plan.
>
> [1]
>
> https://github.com/mapr/drill-test-framework/tree/master/framework/resources/Functional/partition_pruning/dfs/csv/plan
>
> - Rahul
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message