drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abdel Hakim Deneche <adene...@maprtech.com>
Subject Re: [What is the purpose of ExternalSort's MAX_SORT_BYTES
Date Thu, 27 Aug 2015 21:04:47 GMT
the change is part of the fix for DRILL-3555
<https://issues.apache.org/jira/browse/DRILL-3555>

Thx

On Thu, Aug 27, 2015 at 2:00 PM, Khurram Faraaz <kfaraaz@maprtech.com>
wrote:

> Hakim, can you please log a JIRA for this change, so we have a track of why
> this change was made. Thanks.
>
> On Thu, Aug 27, 2015 at 10:30 AM, Abdel Hakim Deneche <
> adeneche@maprtech.com
> > wrote:
>
> > Thanks Steven
> >
> > On Thu, Aug 27, 2015 at 10:19 AM, Steven Phillips <steven@dremio.com>
> > wrote:
> >
> > > I think it probably isn't needed anymore. O believe it is a holdover
> from
> > > before spilling was implemented. It doesn't seem to serve any purpose
> > now.
> > >
> > > On Thu, Aug 27, 2015 at 9:17 AM, Abdel Hakim Deneche <
> > > adeneche@maprtech.com>
> > > wrote:
> > >
> > > > anyone ?
> > > >
> > > > On Tue, Aug 25, 2015 at 2:56 PM, Abdel Hakim Deneche <
> > > > adeneche@maprtech.com>
> > > > wrote:
> > > >
> > > > > When running a window function query on large datasets,
> > > > > increasing planner.memory.max_query_memory_per_node can actually
> help
> > > the
> > > > > query not run out of memory. But in some cases this can cause some
> > > issues
> > > > > (see DRILL-3555 <https://issues.apache.org/jira/browse/DRILL-3555
> >)
> > > > >
> > > > > This seems to be caused by a hardcoded limit in ExternalSort called
> > > > > MAX_SORT_BYTES. What is the purpose of this limit ?
> > > > >
> > > > > --
> > > > >
> > > > > Abdelhakim Deneche
> > > > >
> > > > > Software Engineer
> > > > >
> > > > >   <http://www.mapr.com/>
> > > > >
> > > > >
> > > > > Now Available - Free Hadoop On-Demand Training
> > > > > <
> > > >
> > >
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Abdelhakim Deneche
> > > >
> > > > Software Engineer
> > > >
> > > >   <http://www.mapr.com/>
> > > >
> > > >
> > > > Now Available - Free Hadoop On-Demand Training
> > > > <
> > > >
> > >
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> >
> > Abdelhakim Deneche
> >
> > Software Engineer
> >
> >   <http://www.mapr.com/>
> >
> >
> > Now Available - Free Hadoop On-Demand Training
> > <
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > >
> >
>



-- 

Abdelhakim Deneche

Software Engineer

  <http://www.mapr.com/>


Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message