calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Reshetov <alexander.v.reshe...@gmail.com>
Subject Re: Unclear hard-coding in planner and other parts of source code
Date Fri, 24 Jul 2015 20:36:45 GMT
Thanks for clarification about meaning of such constructions.


Regarding issue with Calcite's cost optimizer:

In my company we have home-brewed column-oriented storage.
To plug our storage to Calcite I implemented TranslatableTable, similar to
CsvTranslatableTable from Calcite's example.

And I stuck with a problem. For example, I have a table 'data' which
contains 20 columns - from X1 to X20.

My query is "select count(X5) from data". I expect that Calcite will choose
plan which reads only this specified column X5. But it reads all 20 columns.
It works with "select X5 from data" as expected - suppose it was jast luck.

To set cost I overrode computeSelfCost method where cpu cost and io cost
was multiplied by length of row. But it didn't help.

After investigation of Calcite's implementation of cost comparison I noticed
that VolcanoCost during comparison ignores everything but number of rows.
Obviously, I can do own implementation of RelOptCost, but, I suppose that
VolcanoCost ignores other values for some reason.

So now I overriding computeSelfCost method like this:

    super.computeSelfCost(planner).multiplyBy(numOfFieldsInProjection);

And Calcite choosing the right plan - reading only one column.

But I'm concerned about this overriding. Because semantic is broken - number
of rows is incorrect now. And it may cause issues in the future. For example
with joins.

On Fri, Jul 24, 2015 at 9:54 PM, Julian Hyde <jhyde.apache@gmail.com> wrote:
> +1
>
> My thinking exactly.
>
>> On Jul 24, 2015, at 11:32 AM, Vladimir Sitnikov <sitnikov.vladimir@gmail.com>
wrote:
>>
>> I'm not sure if that was considered, but the good part with if
>> (true)/if (false) is the "if-commented" code is compiled and it works
>> for "find usages", etc use-cases.
>>
>> It might give an insight "what works and what does not".
>>
>> "If it doesn't worth compiling, it doesn't worth maintaining (or don't
>> check in commented out code)"  via
>> https://twitter.com/mariofusco/status/378068886081769472 from
>> http://www.codemesh.io/codemesh2013/slides/viktor-klang.pdf
>>
>> Vladimir
>

Mime
View raw message