impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars Volker ...@cloudera.com>
Subject Re: Request for feedback: IMPALA-2550 Introduce query-wide execution context.
Date Wed, 20 Jul 2016 17:24:01 GMT
Sure. Currently we duplicate state in fragment instances of a single query,
for example each plan fragment executor creates its own descriptor table,
which could actually be shared among all fragment instances. For RPC
batching (IMPALA-2550) we will transmit the descriptor table only once, and
it seems wasteful to re-build it for every fragment instance. This probably
holds true for other information as well, that is actually the same for
multiple fragment instances of a query, such as runtime filters.

On Wed, Jul 20, 2016 at 7:15 PM, Henry Robinson <henry@cloudera.com> wrote:

> Could you say something about the use cases you're trying to address?
> That'll make it easier to review the design.
>
> On 20 July 2016 at 10:08, Lars Volker <lv@cloudera.com> wrote:
>
> > Hi all,
> >
> > I started drafting a header file for a query-wide QueryExecState class
> > here: https://gerrit.cloudera.org/#/c/3686/2
> >
> > It's not much and I couldn't figure out how much of the overall changes
> > needed for RPC batching should be included in this change. Most notably
> we
> > might want to replace FragmentExecState in FragmentMgr (now QueryExecMgr)
> > with QueryExecState altogether and drive execution from that class. On
> the
> > other hand keeping the QueryExecState as simple as possible for now would
> > make the change less intrusive.
> >
> > I'd be glad for feedback on this first draft.
> >
> > Thanks, Lars
> >
>
>
>
> --
> Henry Robinson
> Software Engineer
> Cloudera
> 415-994-6679
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message