hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "刘奎恩(局外)" <kuien....@alibaba-inc.com>
Subject 回复:a vectorized execution design document
Date Thu, 01 Mar 2018 07:19:51 GMT
Thanks to Shujie for helpful reply.  Yes, it is transparent to upper logics which following Volcano
model to evaluate cost and generate plan. When we finish the vectorization work (mostly),
we may seek for a Vectorization-aware QO, with consider Bach-a-time, or Operatior-a-time,
rather than Tuple-a-time.

-------------——
Kuien Liu/奎恩
------------------------------------------------------------------发件人:Shujie Zhang
<shzhang@pivotal.io>发送时间:2018年2月27日(星期二) 09:41收件人:dev
<dev@hawq.incubator.apache.org>; 刘奎恩(局外) <kuien.lke@alibaba-inc.com>主 题:Re:
a vectorized execution design document
Hi,








We check the plan node to see if it can be vectorized when the Plan has been generated, In
this phase, the only cheapest Plan had been selected, so we have no chance to change it.
If we want to generate the vectorized Plan in the optimizer, we should generate the vectorized
Path and compute the cost of it, then we can compare with both the cost of them and choose
the cheaper one, the trouble is both build-in-optimizer and ORCA should be refactored, it
is a complex work:).  Another trouble is that the solution space of optimizer would become
larger becuase of adding a new type Path, the planning time should be controlled.










In this design, we change the Plan after it was generated,  it is transparent to upper modules, so
the optimizer is also can be changed to fit the current vectorized Plan in the future.
Thanks,Zhang Shujie
On Mon, Feb 26, 2018 at 3:01 PM, 刘奎恩(局外) <kuien.lke@alibaba-inc.com> wrote:
Nice doc, clear design. It is a good start ! I saw an example on aggregation is illustrated
during the doc, we may implement more operators with this design, for example, SORT, JOIN. 

One question is: we implement vectorization under plan three, that is, the optimizer cannot
feel the change in this way, it still estimates overall cost like

' total_cost = startup_cost + cpu_per_tuple * tuples + seq_page_cost * pages 'In my opinion,
the second part (CPU costs) changes a lot, so it is should be a stage design, any further
plan on it? 

-------------——

Kuien Liu/奎恩

------------------------------------------------------------------发件人:Shujie Zhang
<shzhang@pivotal.io>发送时间:2018年2月9日(星期五) 16:35收件人:dev <dev@hawq.incubator.apache.org>主 题:a
vectorized execution design document

Hi,



A vectorized execution design document have been uploaded to the issue#1450:

https://issues.apache.org/jira/browse/HAWQ-1450



Inside the document are a lot of ideas about how to implement a vectorized

executor, We welcome any comments on the content and suggestions for

improvement, thanks.



Zhang Shujie

2018-02-09




Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message