drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Phillips <ste...@dremio.com>
Subject Re: [What is the purpose of ExternalSort's MAX_SORT_BYTES
Date Thu, 27 Aug 2015 17:19:03 GMT
I think it probably isn't needed anymore. O believe it is a holdover from
before spilling was implemented. It doesn't seem to serve any purpose now.

On Thu, Aug 27, 2015 at 9:17 AM, Abdel Hakim Deneche <adeneche@maprtech.com>
wrote:

> anyone ?
>
> On Tue, Aug 25, 2015 at 2:56 PM, Abdel Hakim Deneche <
> adeneche@maprtech.com>
> wrote:
>
> > When running a window function query on large datasets,
> > increasing planner.memory.max_query_memory_per_node can actually help the
> > query not run out of memory. But in some cases this can cause some issues
> > (see DRILL-3555 <https://issues.apache.org/jira/browse/DRILL-3555>)
> >
> > This seems to be caused by a hardcoded limit in ExternalSort called
> > MAX_SORT_BYTES. What is the purpose of this limit ?
> >
> > --
> >
> > Abdelhakim Deneche
> >
> > Software Engineer
> >
> >   <http://www.mapr.com/>
> >
> >
> > Now Available - Free Hadoop On-Demand Training
> > <
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >
> >
>
>
>
> --
>
> Abdelhakim Deneche
>
> Software Engineer
>
>   <http://www.mapr.com/>
>
>
> Now Available - Free Hadoop On-Demand Training
> <
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message