drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Krystal Nguyen <kngu...@maprtech.com>
Subject Re: Time for a 1.5 release?
Date Fri, 29 Jan 2016 16:31:15 GMT
I am running drill-1.4 and is able to run count(*) successfully.

0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter session set
`store.hive.optimize_scan_with_native_readers` = true;
+-------+--------------------------------------------------------+
|  ok   |                        summary                         |
+-------+--------------------------------------------------------+
| true  | store.hive.optimize_scan_with_native_readers updated.  |
+-------+--------------------------------------------------------+
1 row selected (0.374 seconds)
0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select count(*) from
lineitem_parquet_partitioned_hive_string;
+---------+
| EXPR$0  |
+---------+
| 60175   |
+---------+
1 row selected (2.986 seconds)

On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <jinfengni99@gmail.com> wrote:

> Venki and I did some investigation for DRILL-4323. The issue reported
> in DRILL-4323 seems to happen on 1.4.0 release as well. Seems to us
> this is not a regression from 1.4.0; it's a regression from 1.3.0
> probably.
>
> DRILL-4083 makes the planner to use DrillHiveNativeReader in stead of
> HiveReader for "select count(*) from hive_table" query.  However, the
> Project after the scan produces empty schema.  Before DRILL-4083,
> Drill uses HiveScan, which works fine.
>
>
> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
> <altekrusejason@gmail.com> wrote:
> > We could revert the change to the StoragePlugin interface to add the
> > getLogicalOptimizerRules
> > and getPhysicalOptimizerRules, and only put those in
> AbstractStoragePlugin
> > with default implementations.
> >
> > We could then update the docs to tell users to not implement the
> interface
> > directly, and instead extend the abstract class, which I believe would
> > allow us to make changes like this in the future without breaking
> plugins.
> > At a major release version we could then just pull up all of the new
> > methods we added to the abstract class into the interface, or just get
> rid
> > of it all-together.
> >
> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
> venki.korukanti@gmail.com>
> > wrote:
> >
> >> Here is the partial fix:
> >>
> >>
> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
> >>
> >> If the existing StragePlugins are implementing AbstractStoragePlugin
> class
> >> this fix should resolve the backward compat issue. If they are
> implementing
> >> StoragePlugin interface directly then it won't resolve the issue. Not
> sure
> >> how to handle it in that case. Let me know if there any ways to resolve
> >> this issue completely.
> >>
> >> Thanks
> >> Venki
> >>
> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <asinha@maprtech.com>
> wrote:
> >>
> >> > I was able to do the build with maven 3.3.9 (after doing 'brew install
> >> > maven').  Here are my env variables related to maven:
> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
> >> >
> >> >
> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <jacques@dremio.com>
> >> > wrote:
> >> >
> >> > > Aman, for reference on the Maven version issue. You can see the
> output
> >> of
> >> > > the shade plugin between the two Apache Builds. If you search for
> >> > > maven-shade-plugin you'll see that build 647 (maven 3.0.5) fails
> >> because
> >> > it
> >> > > includes a bunch of hadoop (and other libraries that are listed as
> >> being
> >> > > excluded). If you look at build 648 (maven 3.3.3), those items are
> >> > included
> >> > > so that the build completes successfully.
> >> > >
> >> > > https://builds.apache.org/job/drill-scm/647/consoleText
> >> > > https://builds.apache.org/job/drill-scm/648/consoleText
> >> > >
> >> > > --
> >> > > Jacques Nadeau
> >> > > CTO and Co-Founder, Dremio
> >> > >
> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
> >> > > challapallirahul@gmail.com> wrote:
> >> > >
> >> > > > Jason,
> >> > > >
> >> > > > As of now I haven't narrowed down the exact commit which caused
> >> > > DRILL-4323
> >> > > > and I do not know the scope of the fix.
> >> > > >
> >> > > > - Rahul
> >> > > >
> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
> >> > > altekrusejason@gmail.com
> >> > > > >
> >> > > > wrote:
> >> > > >
> >> > > > > I think it makes sense to wait for Venki's fix of the API,
as
> >> > > releasing a
> >> > > > > version with the change would make reverting/refactoring
the API
> >> > > breaking
> >> > > > > change less useful.
> >> > > > >
> >> > > > > Do you have any idea about the scope of fixing the hive
native
> >> > reader,
> >> > > or
> >> > > > > which change since the 1.4 release broke it? Is someone
> available
> >> to
> >> > > look
> >> > > > > at it today?
> >> > > > >
> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
> >> > > > > challapallirahul@gmail.com> wrote:
> >> > > > >
> >> > > > > > Just found an issue with hive native parquet reader
> (DRILL-4323).
> >> > > This
> >> > > > > is a
> >> > > > > > regression from 1.4 and in my opinion should be treated
as a
> >> > blocker.
> >> > > > > >
> >> > > > > > - Rahul
> >> > > > > >
> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
> >> > > > > > venki.korukanti@gmail.com>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Sorry I didn't realize I was breaking the public
interface
> API.
> >> > Let
> >> > > > me
> >> > > > > > work
> >> > > > > > > on a patch to make it backward compatible.
> >> > > > > > >
> >> > > > > > > Thanks
> >> > > > > > > Venki
> >> > > > > > >
> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips
<
> >> > > steven@dremio.com
> >> > > > >
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > > > I just wanted to bring up an issue that I
just now
> >> discovered,
> >> > > that
> >> > > > > has
> >> > > > > > > > caused me a fair amount of grief.
> >> > > > > > > >
> >> > > > > > > > https://github.com/apache/drill/pull/300/commits
> >> > > > > > > >
> >> > > > > > > > DRILL-4198 changes a user-facing API, and
causes
> >> StoragePlugins
> >> > > > that
> >> > > > > > were
> >> > > > > > > > compiled against currently released versions
of Drill to
> no
> >> > > longer
> >> > > > > > > > functional properly. I would prefer that
this breaking
> change
> >> > be
> >> > > > > > modified
> >> > > > > > > > to be backward compatible if possible.
> >> > > > > > > >
> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse
<
> >> > > > > > > > altekrusejason@gmail.com>
> >> > > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > Hi Aman,
> >> > > > > > > > >
> >> > > > > > > > > This is the failure that he was seeing.
He figured out
> that
> >> > the
> >> > > > new
> >> > > > > > > > > exclusions in jdbc-all were not being
respected when the
> >> > build
> >> > > > was
> >> > > > > > run
> >> > > > > > > > with
> >> > > > > > > > > an older Maven version, causing the
jar size to increase
> >> > > > > > significantly.
> >> > > > > > > > He
> >> > > > > > > > > added an enforcer to make sure the JAR
didn't grow
> >> > > unexpectedly.
> >> > > > > Can
> >> > > > > > > you
> >> > > > > > > > > try to update your maven version and
re-run the build?
> >> > > > > > > > >
> >> > > > > > > > > - Jason
> >> > > > > > > > >
> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman
Sinha <
> >> > > > asinha@maprtech.com>
> >> > > > > > > > wrote:
> >> > > > > > > > >
> >> > > > > > > > > > Jacques, I am getting the following
build failure on
> the
> >> > > latest
> >> > > > > > > master
> >> > > > > > > > > > branch...is this what you saw for
the Apache build ?
> My
> >> > mvn
> >> > > > > > version
> >> > > > > > > > > output
> >> > > > > > > > > > is shown below.  Should we all
be upgrading to a newer
> >> mvn
> >> > ?
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
> >> > > > > > > > > > (enforce-jdbc-jar-compactness)
@ drill-jdbc-all ---
> >> > > > > > > > > > [WARNING] Rule 0:
> >> > > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
> >> > > > > > > > > failed
> >> > > > > > > > > > with message:
> >> > > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar
is outside
> the
> >> > > > > expected
> >> > > > > > > size
> >> > > > > > > > > > range.
> >> > > > > > > > > >
> >> > > > > > > > > >                   This is likely
due to you adding new
> >> > > > > dependencies
> >> > > > > > > to
> >> > > > > > > > a
> >> > > > > > > > > > java-exec and not updating the
excludes in this
> module.
> >> > This
> >> > > is
> >> > > > > > > > important
> >> > > > > > > > > > as it minimizes the size of the
dependency of Drill
> >> > > application
> >> > > > > > > users.
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> > > > > > > > > > size (44664121) too large. Max.
is
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > Administrators-MacBook-Pro-144:incubator-drill
asinha$
> >> mvn
> >> > > > > > --version
> >> > > > > > > > > > Apache Maven 3.0.5
> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
> >> > > > > > > > 2013-02-19
> >> > > > > > > > > > 05:51:28-0800)
> >> > > > > > > > > > Maven home: /opt/local/share/java/maven3
> >> > > > > > > > > > Java version: 1.7.0_45, vendor:
Oracle Corporation
> >> > > > > > > > > > Java home:
> >> > > > > > > > > >
> >> > > > >
> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
> >> > > > > > > > > > Default locale: en_US, platform
encoding: UTF-8
> >> > > > > > > > > > OS name: "mac os x", version: "10.9.5",
arch:
> "x86_64",
> >> > > family:
> >> > > > > > "mac"
> >> > > > > > > > > >
> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM,
Jacques Nadeau <
> >> > > > > > jacques@dremio.com>
> >> > > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > Build back to normal. It looks
like the Apache
> server
> >> was
> >> > > > using
> >> > > > > > an
> >> > > > > > > > old
> >> > > > > > > > > > > version of Maven. Once I switched
to something more
> >> > recent,
> >> > > > the
> >> > > > > > > build
> >> > > > > > > > > > > passed.
> >> > > > > > > > > > >
> >> > > > > > > > > > > --
> >> > > > > > > > > > > Jacques Nadeau
> >> > > > > > > > > > > CTO and Co-Founder, Dremio
> >> > > > > > > > > > >
> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02
AM, Jacques Nadeau <
> >> > > > > > > jacques@dremio.com>
> >> > > > > > > > > > > wrote:
> >> > > > > > > > > > >
> >> > > > > > > > > > > > Hmm... this merge caused
the Apache build to fail.
> >> > > > > > > Investigating...
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > --
> >> > > > > > > > > > > > Jacques Nadeau
> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > On Thu, Jan 28, 2016
at 6:31 AM, Jacques Nadeau <
> >> > > > > > > > jacques@dremio.com>
> >> > > > > > > > > > > > wrote:
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >> I got clean regression
runs as well. I've merged
> the
> >> > > > patch.
> >> > > > > > > > > > > >>
> >> > > > > > > > > > > >> Jason, you want to
start the release process?
> >> > > > > > > > > > > >>
> >> > > > > > > > > > > >> --
> >> > > > > > > > > > > >> Jacques Nadeau
> >> > > > > > > > > > > >> CTO and Co-Founder,
Dremio
> >> > > > > > > > > > > >>
> >> > > > > > > > > > > >> On Wed, Jan 27, 2016
at 10:42 PM, Abhishek
> Girish <
> >> > > > > > > > agirish@mapr.com
> >> > > > > > > > > >
> >> > > > > > > > > > > >> wrote:
> >> > > > > > > > > > > >>
> >> > > > > > > > > > > >>> Had two clean
Functional runs. TPC-H SF100 was
> also
> >> > > > > > successful.
> >> > > > > > > > > > > >>>
> >> > > > > > > > > > > >>> On Wed, Jan 27,
2016 at 10:07 PM, rahul
> >> challapalli <
> >> > > > > > > > > > > >>> challapallirahul@gmail.com>
wrote:
> >> > > > > > > > > > > >>>
> >> > > > > > > > > > > >>> > Kicked off
a functional run with your branch.
> >> Will
> >> > > let
> >> > > > > you
> >> > > > > > > know
> >> > > > > > > > > > once
> >> > > > > > > > > > > it
> >> > > > > > > > > > > >>> > finishes
> >> > > > > > > > > > > >>> >
> >> > > > > > > > > > > >>> > - Rahul
> >> > > > > > > > > > > >>> >
> >> > > > > > > > > > > >>> > On Wed,
Jan 27, 2016 at 9:56 PM, Jacques
> Nadeau <
> >> > > > > > > > > > jacques@dremio.com>
> >> > > > > > > > > > > >>> > wrote:
> >> > > > > > > > > > > >>> >
> >> > > > > > > > > > > >>> > > 4196
was merged today. I have an updated
> patch
> >> > for
> >> > > > 4291
> >> > > > > > > that
> >> > > > > > > > is
> >> > > > > > > > > > > >>> ready.
> >> > > > > > > > > > > >>> > > Unfortunately,
it seems that something isn't
> >> > > working
> >> > > > > with
> >> > > > > > > our
> >> > > > > > > > > > > >>> extended
> >> > > > > > > > > > > >>> > > tests
so I haven't been able to run an
> extended
> >> > > > > > regression.
> >> > > > > > > > > Unit
> >> > > > > > > > > > > >>> tests
> >> > > > > > > > > > > >>> > > pass.
Is someone else possibly able to run a
> >> > > > regression
> >> > > > > > > suite
> >> > > > > > > > > > > against
> >> > > > > > > > > > > >>> > this
> >> > > > > > > > > > > >>> > > branch
[1] so we can confirm things look
> good
> >> and
> >> > > > start
> >> > > > > > the
> >> > > > > > > > > > release
> >> > > > > > > > > > > >>> > > process?
> >> > > > > > > > > > > >>> > >
> >> > > > > > > > > > > >>> > > thanks,
> >> > > > > > > > > > > >>> > > Jacques
> >> > > > > > > > > > > >>> > >
> >> > > > > > > > > > > >>> > > [1]
> >> > > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
> >> > > > > > > > > > > >>> > >
> >> > > > > > > > > > > >>> > > --
> >> > > > > > > > > > > >>> > > Jacques
Nadeau
> >> > > > > > > > > > > >>> > > CTO
and Co-Founder, Dremio
> >> > > > > > > > > > > >>> > >
> >> > > > > > > > > > > >>> > > On
Mon, Jan 25, 2016 at 11:20 AM, Jacques
> >> Nadeau
> >> > <
> >> > > > > > > > > > > jacques@dremio.com
> >> > > > > > > > > > > >>> >
> >> > > > > > > > > > > >>> > > wrote:
> >> > > > > > > > > > > >>> > >
> >> > > > > > > > > > > >>> > > >
I think the main things are 4196 and 4291
> >> > should
> >> > > be
> >> > > > > > > > > completed.
> >> > > > > > > > > > I
> >> > > > > > > > > > > >>> know
> >> > > > > > > > > > > >>> > > Amit
> >> > > > > > > > > > > >>> > > >
was able to reproduce 4196 locally this
> >> weekend
> >> > > so
> >> > > > I
> >> > > > > > > think
> >> > > > > > > > > > we're
> >> > > > > > > > > > > >>> close
> >> > > > > > > > > > > >>> > on
> >> > > > > > > > > > > >>> > > >
that.
> >> > > > > > > > > > > >>> > > >
> >> > > > > > > > > > > >>> > > >
--
> >> > > > > > > > > > > >>> > > >
Jacques Nadeau
> >> > > > > > > > > > > >>> > > >
CTO and Co-Founder, Dremio
> >> > > > > > > > > > > >>> > > >
> >> > > > > > > > > > > >>> > > >
On Mon, Jan 25, 2016 at 10:14 AM, Zelaine
> >> Fong
> >> > <
> >> > > > > > > > > > > zfong@maprtech.com
> >> > > > > > > > > > > >>> >
> >> > > > > > > > > > > >>> > > wrote:
> >> > > > > > > > > > > >>> > > >
> >> > > > > > > > > > > >>> > > >>
Any updates on this?  What's blocking us
> >> from
> >> > > > taking
> >> > > > > > > this
> >> > > > > > > > > to a
> >> > > > > > > > > > > >>> vote --
> >> > > > > > > > > > > >>> > > the
> >> > > > > > > > > > > >>> > > >>
sort merge join issues?
> >> > > > > > > > > > > >>> > > >>
> >> > > > > > > > > > > >>> > > >>
-- Zelaine
> >> > > > > > > > > > > >>> > > >>
> >> > > > > > > > > > > >>> > > >>
On Tue, Jan 19, 2016 at 4:18 PM, Jacques
> >> > Nadeau
> >> > > <
> >> > > > > > > > > > > >>> jacques@dremio.com>
> >> > > > > > > > > > > >>> > > >>
wrote:
> >> > > > > > > > > > > >>> > > >>
> >> > > > > > > > > > > >>> > > >>
> Bumping this thread...
> >> > > > > > > > > > > >>> > > >>
>
> >> > > > > > > > > > > >>> > > >>
> Here are the issues that were
> mentioned in
> >> > > this
> >> > > > > > thread
> >> > > > > > > > > along
> >> > > > > > > > > > > >>> with a
> >> > > > > > > > > > > >>> > > >>
> proposed categorization:
> >> > > > > > > > > > > >>> > > >>
>
> >> > > > > > > > > > > >>> > > >>
> Release Blockers
> >> > > > > > > > > > > >>> > > >>
> In-progress Amit
> >> > > > > > > > > > > >>>
> https://issues.apache.org/jira/browse/DRILL-4190
> >> > > > > > > > > > > >>> > > >>
> In-progress Amit
> >> > > > > > > > > > > >>>
> https://issues.apache.org/jira/browse/DRILL-4196
> >> > > > > > > > > > > >>> > > >>
> Ready to merge Jacques
> >> > > > > > > > > > > >>> > >
> >> https://issues.apache.org/jira/browse/DRILL-4246
> >> > > > > > > > > > > >>> > > >>
> In-review Jinfeng
> >> > > > > > > > > > > >>>
> https://issues.apache.org/jira/browse/DRILL-4256
> >> > > > > > > > > > > >>> > > >>
> In-progress Jacques
> >> > > > > > > > > > > >>> >
> https://issues.apache.org/jira/browse/DRILL-4278
> >> > > > > > > > > > > >>> > > >>
> Ready to merge Laurent
> >> > > > > > > > > > > >>> > >
> >> https://issues.apache.org/jira/browse/DRILL-4285
> >> > > > > > > > > > > >>> > > >>
> Nice to Have
> >> > > > > > > > > > > >>> > > >>
> Open Jason/Hakim
> >> > > > > > > > > > > >>>
> https://issues.apache.org/jira/browse/DRILL-4247
> >> > > > > > > > > > > >>> > > >>
> In-progress Jason
> >> > > > > > > > > > > >>>
> https://issues.apache.org/jira/browse/DRILL-4203
> >> > > > > > > > > > > >>> > > >>
> Open Jacques
> >> > > > > > > > > > https://issues.apache.org/jira/browse/DRILL-4266
> >> > > > > > > > > > > >>> > > >>
> Ready to merge Jacques
> >> > > > > > > > > > > >>> > >
> >> https://issues.apache.org/jira/browse/DRILL-4131
> >> > > > > > > > > > > >>> > > >>
>
> >> > > > > > > > > > > >>> > > >>
> What do others think? Let's try to get
> the
> >> > > > > blockers
> >> > > > > > > > > wrapped
> >> > > > > > > > > > up
> >> > > > > > > > > > > >>> in
> >> > > > > > > > > > > >>> > the
> >> > > > > > > > > > > >>> > > >>
next
> >> > > > > > > > > > > >>> > > >>
> day or two and start a release vote...
> >> > > > > > > > > > > >>> > > >>
>
> >> > > > > > > > > > > >>> > > >>
>
> >> > > > > > > > > > > >>> > > >>
>
> >> > > > > > > > > > > >>> > > >>
> --
> >> > > > > > > > > > > >>> > > >>
> Jacques Nadeau
> >> > > > > > > > > > > >>> > > >>
> CTO and Co-Founder, Dremio
> >> > > > > > > > > > > >>> > > >>
>
> >> > > > > > > > > > > >>> > > >>
> On Mon, Jan 4, 2016 at 1:48 PM, Jason
> >> > > Altekruse
> >> > > > <
> >> > > > > > > > > > > >>> > > >>
altekrusejason@gmail.com>
> >> > > > > > > > > > > >>> > > >>
> wrote:
> >> > > > > > > > > > > >>> > > >>
>
> >> > > > > > > > > > > >>> > > >>
> > Hello All,
> >> > > > > > > > > > > >>> > > >>
> >
> >> > > > > > > > > > > >>> > > >>
> > With the allocator changes merged and
> >> > about
> >> > > a
> >> > > > > > month
> >> > > > > > > > > since
> >> > > > > > > > > > > the
> >> > > > > > > > > > > >>> last
> >> > > > > > > > > > > >>> > > >>
> release
> >> > > > > > > > > > > >>> > > >>
> > I think it would be good to start a
> vote
> >> > > > soon. I
> >> > > > > > > would
> >> > > > > > > > > > like
> >> > > > > > > > > > > to
> >> > > > > > > > > > > >>> > > >>
volunteer
> >> > > > > > > > > > > >>> > > >>
> to
> >> > > > > > > > > > > >>> > > >>
> > be release manager.
> >> > > > > > > > > > > >>> > > >>
> >
> >> > > > > > > > > > > >>> > > >>
> > I know that there were some issues
> that
> >> > were
> >> > > > > > > > identified
> >> > > > > > > > > > > after
> >> > > > > > > > > > > >>> the
> >> > > > > > > > > > > >>> > > >>
> transfer
> >> > > > > > > > > > > >>> > > >>
> > patch was merged. I think that these
> >> > issues
> >> > > > > should
> >> > > > > > > be
> >> > > > > > > > > > fixed
> >> > > > > > > > > > > >>> before
> >> > > > > > > > > > > >>> > > we
> >> > > > > > > > > > > >>> > > >>
> cut a
> >> > > > > > > > > > > >>> > > >>
> > release candidate.
> >> > > > > > > > > > > >>> > > >>
> >
> >> > > > > > > > > > > >>> > > >>
> > From looking at the associated JIRAs
> it
> >> > > looked
> >> > > > > > like
> >> > > > > > > > > there
> >> > > > > > > > > > > was
> >> > > > > > > > > > > >>> a
> >> > > > > > > > > > > >>> > > >>
possible
> >> > > > > > > > > > > >>> > > >>
> > short term fix just adjusting the
> >> > > > > > > > > > max_query_memory_per_node
> >> > > > > > > > > > > >>> > option,
> >> > > > > > > > > > > >>> > > >>
and
> >> > > > > > > > > > > >>> > > >>
> > some more involved work to change
> how we
> >> > > > > determine
> >> > > > > > > the
> >> > > > > > > > > > > correct
> >> > > > > > > > > > > >>> > time
> >> > > > > > > > > > > >>> > > to
> >> > > > > > > > > > > >>> > > >>
> > spill during external sort. I
> believe it
> >> > > makes
> >> > > > > > sense
> >> > > > > > > > to
> >> > > > > > > > > > make
> >> > > > > > > > > > > >>> > > external
> >> > > > > > > > > > > >>> > > >>
> sort
> >> > > > > > > > > > > >>> > > >>
> > work well with the newly improved
> memory
> >> > > > > > accounting
> >> > > > > > > > > before
> >> > > > > > > > > > > >>> > cutting
a
> >> > > > > > > > > > > >>> > > >>
> > release, but I'm not sure how much
> work
> >> is
> >> > > > left
> >> > > > > to
> >> > > > > > > be
> >> > > > > > > > > done
> >> > > > > > > > > > > >>> there.
> >> > > > > > > > > > > >>> > > [1]
> >> > > > > > > > > > > >>> > > >>
> >
> >> > > > > > > > > > > >>> > > >>
> > Please respond with your thoughts on
> a
> >> > > release
> >> > > > > > soon
> >> > > > > > > > and
> >> > > > > > > > > > any
> >> > > > > > > > > > > >>> JIRAs
> >> > > > > > > > > > > >>> > > you
> >> > > > > > > > > > > >>> > > >>
> would
> >> > > > > > > > > > > >>> > > >>
> > like to include in the release.
> >> > > > > > > > > > > >>> > > >>
> >
> >> > > > > > > > > > > >>> > > >>
> > [1] -
> >> > > > > > > > https://issues.apache.org/jira/browse/DRILL-4243
> >> > > > > > > > > > > >>> > > >>
> >
> >> > > > > > > > > > > >>> > > >>
> > Thanks,
> >> > > > > > > > > > > >>> > > >>
> > Jason
> >> > > > > > > > > > > >>> > > >>
> >
> >> > > > > > > > > > > >>> > > >>
>
> >> > > > > > > > > > > >>> > > >>
> >> > > > > > > > > > > >>> > > >
> >> > > > > > > > > > > >>> > > >
> >> > > > > > > > > > > >>> > >
> >> > > > > > > > > > > >>> >
> >> > > > > > > > > > > >>>
> >> > > > > > > > > > > >>
> >> > > > > > > > > > > >>
> >> > > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message