drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hanifi GUNES <hanifigu...@gmail.com>
Subject Re: setting planner max width
Date Wed, 28 Oct 2015 23:18:28 GMT
Yup. That makes sense. All ~150 minor fragments seem under a sing;e major
fragment though. I should dig in further to see what's going on here.

-H+

2015-10-28 16:14 GMT-07:00 Jacques Nadeau <jacques@dremio.com>:

> Max width per node is per major fragment per node, not per query.
>
> So you should see no more than 60 minor fragments for any particular major
> fragment.
>
> Remember that in most cases, a multi-major-fragment query has blocking
> operations in it.
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Wed, Oct 28, 2015 at 4:11 PM, Hanifi GUNES <hg@apache.org> wrote:
>
> > On a 10 node cluster, I am executing a query with the following
> >
> > *alter session set `planner.width.max_per_node`=6;*
> >
> > and see 153 minor fragments reported in the profiles tab whereas I would
> > expect a max parallelization of 60 cluster-wide.
> >
> > Is not this option bounding the max # of threads per query per node?
> Need a
> > second look here.
> >
> >
> > Thanks.
> > -Hanifi
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message