Mailing-List: contact dev-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@impala.incubator.apache.org
MIME-Version: 1.0
From: yu feng <olaptestyu@gmail.com>
Date: Fri, 3 Nov 2017 14:32:43 +0800
Message-ID: <CAPprPdSvHTpZxsXv-bdHB5o-2Yfy1jTV6iku_Y-R+Ca-jrhDjQ@mail.gmail.com>
Subject: extend impala to query from Elasticsearch
To: dev@impala.incubator.apache.org
Content-Type: multipart/alternative; boundary="94eb2c0de9d6b59378055d0e45fa"
archived-at: Fri, 03 Nov 2017 06:32:50 -0000

--94eb2c0de9d6b59378055d0e45fa
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi All =EF=BC=9A

   We are try to query data from Elasticsearch using impala, we want
to take advantage of fast speed of impala engine and fast filter and
aggregation speed of Elasticsearch.

I want to do it in the following way :

1=E3=80=81add a new Table type=EF=BC=88metadata=EF=BC=89 called ES Table.
2=E3=80=81add two new ExecNode(ESScanNode and ESAggregation) to implements =
query to
ES.
3=E3=80=81when a query to ES Table, try to rewrite execution plan while con=
tains
Aggregation(parent) and ESScanNode(child) to a ESAggregation.

In this way, I think it can scan and do aggregation by ES.

I want to know what attitude about the combination, and Is it some better
way to implement it ?

Thanks a lot.

--94eb2c0de9d6b59378055d0e45fa--