drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jinfeng Ni <jinfengn...@gmail.com>
Subject Re: Time for a 1.5 release?
Date Fri, 29 Jan 2016 21:53:40 GMT
Sean probably is quite close to find a fix for this issue. Let's see
whether he can post a patch soon.


On Fri, Jan 29, 2016 at 1:29 PM, rahul challapalli
<challapallirahul@gmail.com> wrote:
> DRILL-4323 only happens when assertions are turned on (my install script
> automatically enables assertions).
>
> - Rahul
>
> On Fri, Jan 29, 2016 at 12:40 PM, Zelaine Fong <zfong@maprtech.com> wrote:
>
>> Rahul,
>>
>> Let us know if you can reproduce the issue with assertions turned OFF.  If
>> you cannot, then I think this should not be a blocker for 1.5, and Hsuan
>> will continue to look into fixing this.
>>
>> -- Zelaine
>>
>> On Fri, Jan 29, 2016 at 12:11 PM, Krystal Nguyen <knguyen@maprtech.com>
>> wrote:
>>
>> > I do see the same error with assertion turned on.
>> >
>> > On Fri, Jan 29, 2016 at 8:45 AM, Jinfeng Ni <jinfengni99@gmail.com>
>> wrote:
>> >
>> > > Krystal, by any chance, did you turn on assertion when you start
>> > drillbit?
>> > >
>> > > I think the error in drill-4323 will be raised when assertion = on.
>> > >
>> > >
>> > >
>> > > On Fri, Jan 29, 2016 at 8:31 AM, Krystal Nguyen <knguyen@maprtech.com>
>> > > wrote:
>> > > > I am running drill-1.4 and is able to run count(*) successfully.
>> > > >
>> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> alter session set
>> > > > `store.hive.optimize_scan_with_native_readers` = true;
>> > > > +-------+--------------------------------------------------------+
>> > > > |  ok   |                        summary                         |
>> > > > +-------+--------------------------------------------------------+
>> > > > | true  | store.hive.optimize_scan_with_native_readers updated.  |
>> > > > +-------+--------------------------------------------------------+
>> > > > 1 row selected (0.374 seconds)
>> > > > 0: jdbc:drill:zk=10.10.100.113:5181,10.10.100> select count(*) from
>> > > > lineitem_parquet_partitioned_hive_string;
>> > > > +---------+
>> > > > | EXPR$0  |
>> > > > +---------+
>> > > > | 60175   |
>> > > > +---------+
>> > > > 1 row selected (2.986 seconds)
>> > > >
>> > > > On Fri, Jan 29, 2016 at 8:12 AM, Jinfeng Ni <jinfengni99@gmail.com>
>> > > wrote:
>> > > >
>> > > >> Venki and I did some investigation for DRILL-4323. The issue
>> reported
>> > > >> in DRILL-4323 seems to happen on 1.4.0 release as well. Seems to us
>> > > >> this is not a regression from 1.4.0; it's a regression from 1.3.0
>> > > >> probably.
>> > > >>
>> > > >> DRILL-4083 makes the planner to use DrillHiveNativeReader in stead
>> of
>> > > >> HiveReader for "select count(*) from hive_table" query.  However,
>> the
>> > > >> Project after the scan produces empty schema.  Before DRILL-4083,
>> > > >> Drill uses HiveScan, which works fine.
>> > > >>
>> > > >>
>> > > >> On Thu, Jan 28, 2016 at 3:42 PM, Jason Altekruse
>> > > >> <altekrusejason@gmail.com> wrote:
>> > > >> > We could revert the change to the StoragePlugin interface to add
>> the
>> > > >> > getLogicalOptimizerRules
>> > > >> > and getPhysicalOptimizerRules, and only put those in
>> > > >> AbstractStoragePlugin
>> > > >> > with default implementations.
>> > > >> >
>> > > >> > We could then update the docs to tell users to not implement the
>> > > >> interface
>> > > >> > directly, and instead extend the abstract class, which I believe
>> > would
>> > > >> > allow us to make changes like this in the future without breaking
>> > > >> plugins.
>> > > >> > At a major release version we could then just pull up all of the
>> new
>> > > >> > methods we added to the abstract class into the interface, or just
>> > get
>> > > >> rid
>> > > >> > of it all-together.
>> > > >> >
>> > > >> > On Thu, Jan 28, 2016 at 2:59 PM, Venki Korukanti <
>> > > >> venki.korukanti@gmail.com>
>> > > >> > wrote:
>> > > >> >
>> > > >> >> Here is the partial fix:
>> > > >> >>
>> > > >> >>
>> > > >>
>> > >
>> >
>> https://github.com/vkorukanti/drill/commit/6646b0d08df274a22dff870ad5e6f6914b10fa0b
>> > > >> >>
>> > > >> >> If the existing StragePlugins are implementing
>> > AbstractStoragePlugin
>> > > >> class
>> > > >> >> this fix should resolve the backward compat issue. If they are
>> > > >> implementing
>> > > >> >> StoragePlugin interface directly then it won't resolve the issue.
>> > Not
>> > > >> sure
>> > > >> >> how to handle it in that case. Let me know if there any ways to
>> > > resolve
>> > > >> >> this issue completely.
>> > > >> >>
>> > > >> >> Thanks
>> > > >> >> Venki
>> > > >> >>
>> > > >> >> On Thu, Jan 28, 2016 at 2:44 PM, Aman Sinha <asinha@maprtech.com
>> >
>> > > >> wrote:
>> > > >> >>
>> > > >> >> > I was able to do the build with maven 3.3.9 (after doing 'brew
>> > > install
>> > > >> >> > maven').  Here are my env variables related to maven:
>> > > >> >> > M2_HOME=/usr/local/Cellar/maven/3.3.9/libexec
>> > > >> >> > M2=/usr/local/Cellar/maven/3.3.9/libexec/bin
>> > > >> >> > MAVEN_OPTS=-Xmx2048m -XX:MaxPermSize=256m
>> > > >> >> >
>> > > >> >> >
>> > > >> >> > On Thu, Jan 28, 2016 at 2:20 PM, Jacques Nadeau <
>> > > jacques@dremio.com>
>> > > >> >> > wrote:
>> > > >> >> >
>> > > >> >> > > Aman, for reference on the Maven version issue. You can see
>> the
>> > > >> output
>> > > >> >> of
>> > > >> >> > > the shade plugin between the two Apache Builds. If you search
>> > for
>> > > >> >> > > maven-shade-plugin you'll see that build 647 (maven 3.0.5)
>> > fails
>> > > >> >> because
>> > > >> >> > it
>> > > >> >> > > includes a bunch of hadoop (and other libraries that are
>> listed
>> > > as
>> > > >> >> being
>> > > >> >> > > excluded). If you look at build 648 (maven 3.3.3), those
>> items
>> > > are
>> > > >> >> > included
>> > > >> >> > > so that the build completes successfully.
>> > > >> >> > >
>> > > >> >> > > https://builds.apache.org/job/drill-scm/647/consoleText
>> > > >> >> > > https://builds.apache.org/job/drill-scm/648/consoleText
>> > > >> >> > >
>> > > >> >> > > --
>> > > >> >> > > Jacques Nadeau
>> > > >> >> > > CTO and Co-Founder, Dremio
>> > > >> >> > >
>> > > >> >> > > On Thu, Jan 28, 2016 at 2:01 PM, rahul challapalli <
>> > > >> >> > > challapallirahul@gmail.com> wrote:
>> > > >> >> > >
>> > > >> >> > > > Jason,
>> > > >> >> > > >
>> > > >> >> > > > As of now I haven't narrowed down the exact commit which
>> > caused
>> > > >> >> > > DRILL-4323
>> > > >> >> > > > and I do not know the scope of the fix.
>> > > >> >> > > >
>> > > >> >> > > > - Rahul
>> > > >> >> > > >
>> > > >> >> > > > On Thu, Jan 28, 2016 at 1:39 PM, Jason Altekruse <
>> > > >> >> > > altekrusejason@gmail.com
>> > > >> >> > > > >
>> > > >> >> > > > wrote:
>> > > >> >> > > >
>> > > >> >> > > > > I think it makes sense to wait for Venki's fix of the
>> API,
>> > as
>> > > >> >> > > releasing a
>> > > >> >> > > > > version with the change would make reverting/refactoring
>> > the
>> > > API
>> > > >> >> > > breaking
>> > > >> >> > > > > change less useful.
>> > > >> >> > > > >
>> > > >> >> > > > > Do you have any idea about the scope of fixing the hive
>> > > native
>> > > >> >> > reader,
>> > > >> >> > > or
>> > > >> >> > > > > which change since the 1.4 release broke it? Is someone
>> > > >> available
>> > > >> >> to
>> > > >> >> > > look
>> > > >> >> > > > > at it today?
>> > > >> >> > > > >
>> > > >> >> > > > > On Thu, Jan 28, 2016 at 1:33 PM, rahul challapalli <
>> > > >> >> > > > > challapallirahul@gmail.com> wrote:
>> > > >> >> > > > >
>> > > >> >> > > > > > Just found an issue with hive native parquet reader
>> > > >> (DRILL-4323).
>> > > >> >> > > This
>> > > >> >> > > > > is a
>> > > >> >> > > > > > regression from 1.4 and in my opinion should be treated
>> > as
>> > > a
>> > > >> >> > blocker.
>> > > >> >> > > > > >
>> > > >> >> > > > > > - Rahul
>> > > >> >> > > > > >
>> > > >> >> > > > > > On Thu, Jan 28, 2016 at 1:12 PM, Venki Korukanti <
>> > > >> >> > > > > > venki.korukanti@gmail.com>
>> > > >> >> > > > > > wrote:
>> > > >> >> > > > > >
>> > > >> >> > > > > > > Sorry I didn't realize I was breaking the public
>> > > interface
>> > > >> API.
>> > > >> >> > Let
>> > > >> >> > > > me
>> > > >> >> > > > > > work
>> > > >> >> > > > > > > on a patch to make it backward compatible.
>> > > >> >> > > > > > >
>> > > >> >> > > > > > > Thanks
>> > > >> >> > > > > > > Venki
>> > > >> >> > > > > > >
>> > > >> >> > > > > > > On Thu, Jan 28, 2016 at 12:54 PM, Steven Phillips <
>> > > >> >> > > steven@dremio.com
>> > > >> >> > > > >
>> > > >> >> > > > > > > wrote:
>> > > >> >> > > > > > >
>> > > >> >> > > > > > > > I just wanted to bring up an issue that I just now
>> > > >> >> discovered,
>> > > >> >> > > that
>> > > >> >> > > > > has
>> > > >> >> > > > > > > > caused me a fair amount of grief.
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > > > https://github.com/apache/drill/pull/300/commits
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > > > DRILL-4198 changes a user-facing API, and causes
>> > > >> >> StoragePlugins
>> > > >> >> > > > that
>> > > >> >> > > > > > were
>> > > >> >> > > > > > > > compiled against currently released versions of
>> Drill
>> > > to
>> > > >> no
>> > > >> >> > > longer
>> > > >> >> > > > > > > > functional properly. I would prefer that this
>> > breaking
>> > > >> change
>> > > >> >> > be
>> > > >> >> > > > > > modified
>> > > >> >> > > > > > > > to be backward compatible if possible.
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > > > On Thu, Jan 28, 2016 at 11:23 AM, Jason Altekruse <
>> > > >> >> > > > > > > > altekrusejason@gmail.com>
>> > > >> >> > > > > > > > wrote:
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > > > > Hi Aman,
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > > > This is the failure that he was seeing. He
>> figured
>> > > out
>> > > >> that
>> > > >> >> > the
>> > > >> >> > > > new
>> > > >> >> > > > > > > > > exclusions in jdbc-all were not being respected
>> > when
>> > > the
>> > > >> >> > build
>> > > >> >> > > > was
>> > > >> >> > > > > > run
>> > > >> >> > > > > > > > with
>> > > >> >> > > > > > > > > an older Maven version, causing the jar size to
>> > > increase
>> > > >> >> > > > > > significantly.
>> > > >> >> > > > > > > > He
>> > > >> >> > > > > > > > > added an enforcer to make sure the JAR didn't
>> grow
>> > > >> >> > > unexpectedly.
>> > > >> >> > > > > Can
>> > > >> >> > > > > > > you
>> > > >> >> > > > > > > > > try to update your maven version and re-run the
>> > > build?
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > > > - Jason
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > > > On Thu, Jan 28, 2016 at 11:18 AM, Aman Sinha <
>> > > >> >> > > > asinha@maprtech.com>
>> > > >> >> > > > > > > > wrote:
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > > > > Jacques, I am getting the following build
>> failure
>> > > on
>> > > >> the
>> > > >> >> > > latest
>> > > >> >> > > > > > > master
>> > > >> >> > > > > > > > > > branch...is this what you saw for the Apache
>> > build
>> > > ?
>> > > >> My
>> > > >> >> > mvn
>> > > >> >> > > > > > version
>> > > >> >> > > > > > > > > output
>> > > >> >> > > > > > > > > > is shown below.  Should we all be upgrading to
>> a
>> > > newer
>> > > >> >> mvn
>> > > >> >> > ?
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > > [INFO] --- maven-enforcer-plugin:1.3.1:enforce
>> > > >> >> > > > > > > > > > (enforce-jdbc-jar-compactness) @ drill-jdbc-all
>> > ---
>> > > >> >> > > > > > > > > > [WARNING] Rule 0:
>> > > >> >> > > > > > org.apache.maven.plugins.enforcer.RequireFilesSize
>> > > >> >> > > > > > > > > failed
>> > > >> >> > > > > > > > > > with message:
>> > > >> >> > > > > > > > > > The file drill-jdbc-all-1.5.0-SNAPSHOT.jar is
>> > > outside
>> > > >> the
>> > > >> >> > > > > expected
>> > > >> >> > > > > > > size
>> > > >> >> > > > > > > > > > range.
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > >                   This is likely due to you
>> > adding
>> > > new
>> > > >> >> > > > > dependencies
>> > > >> >> > > > > > > to
>> > > >> >> > > > > > > > a
>> > > >> >> > > > > > > > > > java-exec and not updating the excludes in this
>> > > >> module.
>> > > >> >> > This
>> > > >> >> > > is
>> > > >> >> > > > > > > > important
>> > > >> >> > > > > > > > > > as it minimizes the size of the dependency of
>> > Drill
>> > > >> >> > > application
>> > > >> >> > > > > > > users.
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > >
>> > > >> >> > > > > >
>> > > >> >> > > > >
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> /Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> > > >> >> > > > > > > > > > size (44664121) too large. Max. is
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > >
>> > > >> >> > > > > >
>> > > >> >> > > > >
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> 20000000/Users/asinha/incubator-drill/exec/jdbc-all/target/drill-jdbc-all-1.5.0-SNAPSHOT.jar
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > > Administrators-MacBook-Pro-144:incubator-drill
>> > > asinha$
>> > > >> >> mvn
>> > > >> >> > > > > > --version
>> > > >> >> > > > > > > > > > Apache Maven 3.0.5
>> > > >> >> > > (r01de14724cdef164cd33c7c8c2fe155faf9602da;
>> > > >> >> > > > > > > > 2013-02-19
>> > > >> >> > > > > > > > > > 05:51:28-0800)
>> > > >> >> > > > > > > > > > Maven home: /opt/local/share/java/maven3
>> > > >> >> > > > > > > > > > Java version: 1.7.0_45, vendor: Oracle
>> > Corporation
>> > > >> >> > > > > > > > > > Java home:
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > >
>> > > >> /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre
>> > > >> >> > > > > > > > > > Default locale: en_US, platform encoding: UTF-8
>> > > >> >> > > > > > > > > > OS name: "mac os x", version: "10.9.5", arch:
>> > > >> "x86_64",
>> > > >> >> > > family:
>> > > >> >> > > > > > "mac"
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > > On Thu, Jan 28, 2016 at 8:20 AM, Jacques
>> Nadeau <
>> > > >> >> > > > > > jacques@dremio.com>
>> > > >> >> > > > > > > > > > wrote:
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > > > Build back to normal. It looks like the
>> Apache
>> > > >> server
>> > > >> >> was
>> > > >> >> > > > using
>> > > >> >> > > > > > an
>> > > >> >> > > > > > > > old
>> > > >> >> > > > > > > > > > > version of Maven. Once I switched to
>> something
>> > > more
>> > > >> >> > recent,
>> > > >> >> > > > the
>> > > >> >> > > > > > > build
>> > > >> >> > > > > > > > > > > passed.
>> > > >> >> > > > > > > > > > >
>> > > >> >> > > > > > > > > > > --
>> > > >> >> > > > > > > > > > > Jacques Nadeau
>> > > >> >> > > > > > > > > > > CTO and Co-Founder, Dremio
>> > > >> >> > > > > > > > > > >
>> > > >> >> > > > > > > > > > > On Thu, Jan 28, 2016 at 7:02 AM, Jacques
>> > Nadeau <
>> > > >> >> > > > > > > jacques@dremio.com>
>> > > >> >> > > > > > > > > > > wrote:
>> > > >> >> > > > > > > > > > >
>> > > >> >> > > > > > > > > > > > Hmm... this merge caused the Apache build
>> to
>> > > fail.
>> > > >> >> > > > > > > Investigating...
>> > > >> >> > > > > > > > > > > >
>> > > >> >> > > > > > > > > > > > --
>> > > >> >> > > > > > > > > > > > Jacques Nadeau
>> > > >> >> > > > > > > > > > > > CTO and Co-Founder, Dremio
>> > > >> >> > > > > > > > > > > >
>> > > >> >> > > > > > > > > > > > On Thu, Jan 28, 2016 at 6:31 AM, Jacques
>> > > Nadeau <
>> > > >> >> > > > > > > > jacques@dremio.com>
>> > > >> >> > > > > > > > > > > > wrote:
>> > > >> >> > > > > > > > > > > >
>> > > >> >> > > > > > > > > > > >> I got clean regression runs as well. I've
>> > > merged
>> > > >> the
>> > > >> >> > > > patch.
>> > > >> >> > > > > > > > > > > >>
>> > > >> >> > > > > > > > > > > >> Jason, you want to start the release
>> > process?
>> > > >> >> > > > > > > > > > > >>
>> > > >> >> > > > > > > > > > > >> --
>> > > >> >> > > > > > > > > > > >> Jacques Nadeau
>> > > >> >> > > > > > > > > > > >> CTO and Co-Founder, Dremio
>> > > >> >> > > > > > > > > > > >>
>> > > >> >> > > > > > > > > > > >> On Wed, Jan 27, 2016 at 10:42 PM, Abhishek
>> > > >> Girish <
>> > > >> >> > > > > > > > agirish@mapr.com
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > > > > >> wrote:
>> > > >> >> > > > > > > > > > > >>
>> > > >> >> > > > > > > > > > > >>> Had two clean Functional runs. TPC-H
>> SF100
>> > > was
>> > > >> also
>> > > >> >> > > > > > successful.
>> > > >> >> > > > > > > > > > > >>>
>> > > >> >> > > > > > > > > > > >>> On Wed, Jan 27, 2016 at 10:07 PM, rahul
>> > > >> >> challapalli <
>> > > >> >> > > > > > > > > > > >>> challapallirahul@gmail.com> wrote:
>> > > >> >> > > > > > > > > > > >>>
>> > > >> >> > > > > > > > > > > >>> > Kicked off a functional run with your
>> > > branch.
>> > > >> >> Will
>> > > >> >> > > let
>> > > >> >> > > > > you
>> > > >> >> > > > > > > know
>> > > >> >> > > > > > > > > > once
>> > > >> >> > > > > > > > > > > it
>> > > >> >> > > > > > > > > > > >>> > finishes
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> >> > > > > > > > > > > >>> > - Rahul
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> >> > > > > > > > > > > >>> > On Wed, Jan 27, 2016 at 9:56 PM,
>> Jacques
>> > > >> Nadeau <
>> > > >> >> > > > > > > > > > jacques@dremio.com>
>> > > >> >> > > > > > > > > > > >>> > wrote:
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> >> > > > > > > > > > > >>> > > 4196 was merged today. I have an
>> > updated
>> > > >> patch
>> > > >> >> > for
>> > > >> >> > > > 4291
>> > > >> >> > > > > > > that
>> > > >> >> > > > > > > > is
>> > > >> >> > > > > > > > > > > >>> ready.
>> > > >> >> > > > > > > > > > > >>> > > Unfortunately, it seems that
>> something
>> > > isn't
>> > > >> >> > > working
>> > > >> >> > > > > with
>> > > >> >> > > > > > > our
>> > > >> >> > > > > > > > > > > >>> extended
>> > > >> >> > > > > > > > > > > >>> > > tests so I haven't been able to run
>> an
>> > > >> extended
>> > > >> >> > > > > > regression.
>> > > >> >> > > > > > > > > Unit
>> > > >> >> > > > > > > > > > > >>> tests
>> > > >> >> > > > > > > > > > > >>> > > pass. Is someone else possibly able
>> to
>> > > run a
>> > > >> >> > > > regression
>> > > >> >> > > > > > > suite
>> > > >> >> > > > > > > > > > > against
>> > > >> >> > > > > > > > > > > >>> > this
>> > > >> >> > > > > > > > > > > >>> > > branch [1] so we can confirm things
>> > look
>> > > >> good
>> > > >> >> and
>> > > >> >> > > > start
>> > > >> >> > > > > > the
>> > > >> >> > > > > > > > > > release
>> > > >> >> > > > > > > > > > > >>> > > process?
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> > > > > > > > > > > >>> > > thanks,
>> > > >> >> > > > > > > > > > > >>> > > Jacques
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> > > > > > > > > > > >>> > > [1]
>> > > >> >> > > > > https://github.com/jacques-n/drill/tree/DRILL-4291v2
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> > > > > > > > > > > >>> > > --
>> > > >> >> > > > > > > > > > > >>> > > Jacques Nadeau
>> > > >> >> > > > > > > > > > > >>> > > CTO and Co-Founder, Dremio
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> > > > > > > > > > > >>> > > On Mon, Jan 25, 2016 at 11:20 AM,
>> > Jacques
>> > > >> >> Nadeau
>> > > >> >> > <
>> > > >> >> > > > > > > > > > > jacques@dremio.com
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> >> > > > > > > > > > > >>> > > wrote:
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> > > > > > > > > > > >>> > > > I think the main things are 4196
>> and
>> > > 4291
>> > > >> >> > should
>> > > >> >> > > be
>> > > >> >> > > > > > > > > completed.
>> > > >> >> > > > > > > > > > I
>> > > >> >> > > > > > > > > > > >>> know
>> > > >> >> > > > > > > > > > > >>> > > Amit
>> > > >> >> > > > > > > > > > > >>> > > > was able to reproduce 4196 locally
>> > this
>> > > >> >> weekend
>> > > >> >> > > so
>> > > >> >> > > > I
>> > > >> >> > > > > > > think
>> > > >> >> > > > > > > > > > we're
>> > > >> >> > > > > > > > > > > >>> close
>> > > >> >> > > > > > > > > > > >>> > on
>> > > >> >> > > > > > > > > > > >>> > > > that.
>> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> >> > > > > > > > > > > >>> > > > --
>> > > >> >> > > > > > > > > > > >>> > > > Jacques Nadeau
>> > > >> >> > > > > > > > > > > >>> > > > CTO and Co-Founder, Dremio
>> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> >> > > > > > > > > > > >>> > > > On Mon, Jan 25, 2016 at 10:14 AM,
>> > > Zelaine
>> > > >> >> Fong
>> > > >> >> > <
>> > > >> >> > > > > > > > > > > zfong@maprtech.com
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> >> > > > > > > > > > > >>> > > wrote:
>> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> >> > > > > > > > > > > >>> > > >> Any updates on this?  What's
>> > blocking
>> > > us
>> > > >> >> from
>> > > >> >> > > > taking
>> > > >> >> > > > > > > this
>> > > >> >> > > > > > > > > to a
>> > > >> >> > > > > > > > > > > >>> vote --
>> > > >> >> > > > > > > > > > > >>> > > the
>> > > >> >> > > > > > > > > > > >>> > > >> sort merge join issues?
>> > > >> >> > > > > > > > > > > >>> > > >>
>> > > >> >> > > > > > > > > > > >>> > > >> -- Zelaine
>> > > >> >> > > > > > > > > > > >>> > > >>
>> > > >> >> > > > > > > > > > > >>> > > >> On Tue, Jan 19, 2016 at 4:18 PM,
>> > > Jacques
>> > > >> >> > Nadeau
>> > > >> >> > > <
>> > > >> >> > > > > > > > > > > >>> jacques@dremio.com>
>> > > >> >> > > > > > > > > > > >>> > > >> wrote:
>> > > >> >> > > > > > > > > > > >>> > > >>
>> > > >> >> > > > > > > > > > > >>> > > >> > Bumping this thread...
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> > Here are the issues that were
>> > > >> mentioned in
>> > > >> >> > > this
>> > > >> >> > > > > > thread
>> > > >> >> > > > > > > > > along
>> > > >> >> > > > > > > > > > > >>> with a
>> > > >> >> > > > > > > > > > > >>> > > >> > proposed categorization:
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> > Release Blockers
>> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
>> > > >> >> > > > > > > > > > > >>>
>> > > >> https://issues.apache.org/jira/browse/DRILL-4190
>> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Amit
>> > > >> >> > > > > > > > > > > >>>
>> > > >> https://issues.apache.org/jira/browse/DRILL-4196
>> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> https://issues.apache.org/jira/browse/DRILL-4246
>> > > >> >> > > > > > > > > > > >>> > > >> > In-review Jinfeng
>> > > >> >> > > > > > > > > > > >>>
>> > > >> https://issues.apache.org/jira/browse/DRILL-4256
>> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jacques
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> https://issues.apache.org/jira/browse/DRILL-4278
>> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Laurent
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> https://issues.apache.org/jira/browse/DRILL-4285
>> > > >> >> > > > > > > > > > > >>> > > >> > Nice to Have
>> > > >> >> > > > > > > > > > > >>> > > >> > Open Jason/Hakim
>> > > >> >> > > > > > > > > > > >>>
>> > > >> https://issues.apache.org/jira/browse/DRILL-4247
>> > > >> >> > > > > > > > > > > >>> > > >> > In-progress Jason
>> > > >> >> > > > > > > > > > > >>>
>> > > >> https://issues.apache.org/jira/browse/DRILL-4203
>> > > >> >> > > > > > > > > > > >>> > > >> > Open Jacques
>> > > >> >> > > > > > > > > >
>> https://issues.apache.org/jira/browse/DRILL-4266
>> > > >> >> > > > > > > > > > > >>> > > >> > Ready to merge Jacques
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> https://issues.apache.org/jira/browse/DRILL-4131
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> > What do others think? Let's try
>> to
>> > > get
>> > > >> the
>> > > >> >> > > > > blockers
>> > > >> >> > > > > > > > > wrapped
>> > > >> >> > > > > > > > > > up
>> > > >> >> > > > > > > > > > > >>> in
>> > > >> >> > > > > > > > > > > >>> > the
>> > > >> >> > > > > > > > > > > >>> > > >> next
>> > > >> >> > > > > > > > > > > >>> > > >> > day or two and start a release
>> > > vote...
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> > --
>> > > >> >> > > > > > > > > > > >>> > > >> > Jacques Nadeau
>> > > >> >> > > > > > > > > > > >>> > > >> > CTO and Co-Founder, Dremio
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> > On Mon, Jan 4, 2016 at 1:48 PM,
>> > > Jason
>> > > >> >> > > Altekruse
>> > > >> >> > > > <
>> > > >> >> > > > > > > > > > > >>> > > >> altekrusejason@gmail.com>
>> > > >> >> > > > > > > > > > > >>> > > >> > wrote:
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >> > > Hello All,
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> > > With the allocator changes
>> > merged
>> > > and
>> > > >> >> > about
>> > > >> >> > > a
>> > > >> >> > > > > > month
>> > > >> >> > > > > > > > > since
>> > > >> >> > > > > > > > > > > the
>> > > >> >> > > > > > > > > > > >>> last
>> > > >> >> > > > > > > > > > > >>> > > >> > release
>> > > >> >> > > > > > > > > > > >>> > > >> > > I think it would be good to
>> > start
>> > > a
>> > > >> vote
>> > > >> >> > > > soon. I
>> > > >> >> > > > > > > would
>> > > >> >> > > > > > > > > > like
>> > > >> >> > > > > > > > > > > to
>> > > >> >> > > > > > > > > > > >>> > > >> volunteer
>> > > >> >> > > > > > > > > > > >>> > > >> > to
>> > > >> >> > > > > > > > > > > >>> > > >> > > be release manager.
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> > > I know that there were some
>> > issues
>> > > >> that
>> > > >> >> > were
>> > > >> >> > > > > > > > identified
>> > > >> >> > > > > > > > > > > after
>> > > >> >> > > > > > > > > > > >>> the
>> > > >> >> > > > > > > > > > > >>> > > >> > transfer
>> > > >> >> > > > > > > > > > > >>> > > >> > > patch was merged. I think that
>> > > these
>> > > >> >> > issues
>> > > >> >> > > > > should
>> > > >> >> > > > > > > be
>> > > >> >> > > > > > > > > > fixed
>> > > >> >> > > > > > > > > > > >>> before
>> > > >> >> > > > > > > > > > > >>> > > we
>> > > >> >> > > > > > > > > > > >>> > > >> > cut a
>> > > >> >> > > > > > > > > > > >>> > > >> > > release candidate.
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> > > From looking at the associated
>> > > JIRAs
>> > > >> it
>> > > >> >> > > looked
>> > > >> >> > > > > > like
>> > > >> >> > > > > > > > > there
>> > > >> >> > > > > > > > > > > was
>> > > >> >> > > > > > > > > > > >>> a
>> > > >> >> > > > > > > > > > > >>> > > >> possible
>> > > >> >> > > > > > > > > > > >>> > > >> > > short term fix just adjusting
>> > the
>> > > >> >> > > > > > > > > > max_query_memory_per_node
>> > > >> >> > > > > > > > > > > >>> > option,
>> > > >> >> > > > > > > > > > > >>> > > >> and
>> > > >> >> > > > > > > > > > > >>> > > >> > > some more involved work to
>> > change
>> > > >> how we
>> > > >> >> > > > > determine
>> > > >> >> > > > > > > the
>> > > >> >> > > > > > > > > > > correct
>> > > >> >> > > > > > > > > > > >>> > time
>> > > >> >> > > > > > > > > > > >>> > > to
>> > > >> >> > > > > > > > > > > >>> > > >> > > spill during external sort. I
>> > > >> believe it
>> > > >> >> > > makes
>> > > >> >> > > > > > sense
>> > > >> >> > > > > > > > to
>> > > >> >> > > > > > > > > > make
>> > > >> >> > > > > > > > > > > >>> > > external
>> > > >> >> > > > > > > > > > > >>> > > >> > sort
>> > > >> >> > > > > > > > > > > >>> > > >> > > work well with the newly
>> > improved
>> > > >> memory
>> > > >> >> > > > > > accounting
>> > > >> >> > > > > > > > > before
>> > > >> >> > > > > > > > > > > >>> > cutting a
>> > > >> >> > > > > > > > > > > >>> > > >> > > release, but I'm not sure how
>> > much
>> > > >> work
>> > > >> >> is
>> > > >> >> > > > left
>> > > >> >> > > > > to
>> > > >> >> > > > > > > be
>> > > >> >> > > > > > > > > done
>> > > >> >> > > > > > > > > > > >>> there.
>> > > >> >> > > > > > > > > > > >>> > > [1]
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> > > Please respond with your
>> > thoughts
>> > > on
>> > > >> a
>> > > >> >> > > release
>> > > >> >> > > > > > soon
>> > > >> >> > > > > > > > and
>> > > >> >> > > > > > > > > > any
>> > > >> >> > > > > > > > > > > >>> JIRAs
>> > > >> >> > > > > > > > > > > >>> > > you
>> > > >> >> > > > > > > > > > > >>> > > >> > would
>> > > >> >> > > > > > > > > > > >>> > > >> > > like to include in the
>> release.
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> > > [1] -
>> > > >> >> > > > > > > > https://issues.apache.org/jira/browse/DRILL-4243
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> > > Thanks,
>> > > >> >> > > > > > > > > > > >>> > > >> > > Jason
>> > > >> >> > > > > > > > > > > >>> > > >> > >
>> > > >> >> > > > > > > > > > > >>> > > >> >
>> > > >> >> > > > > > > > > > > >>> > > >>
>> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> >> > > > > > > > > > > >>> > > >
>> > > >> >> > > > > > > > > > > >>> > >
>> > > >> >> > > > > > > > > > > >>> >
>> > > >> >> > > > > > > > > > > >>>
>> > > >> >> > > > > > > > > > > >>
>> > > >> >> > > > > > > > > > > >>
>> > > >> >> > > > > > > > > > > >
>> > > >> >> > > > > > > > > > >
>> > > >> >> > > > > > > > > >
>> > > >> >> > > > > > > > >
>> > > >> >> > > > > > > >
>> > > >> >> > > > > > >
>> > > >> >> > > > > >
>> > > >> >> > > > >
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>>

Mime
View raw message