drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arina Yelchiyeva <arina.yelchiy...@gmail.com>
Subject Re: [DISCUSS] Drill 1.11.0 release
Date Thu, 13 Jul 2017 11:46:53 GMT
*Blockers*
DRILL-5659 - problem is identified, Parth is working on the fix.

*Not blockers*
DRILL-5660 - Vitalii
DRILL-5468 - Jinfeng

*Possible blockers*
DRILL-5669 - Boaz Ben-Zvi / Paul
NPE on CSV - Paul

Kind regards
Arina


On Thu, Jul 13, 2017 at 1:28 AM, Paul Rogers <progers@mapr.com> wrote:

> I withdraw my suggestion to fix DRILL-5660. Was a big hassle for us to
> track down the incompatibility the issue internally, but production users
> probably don’t attempt to use one build of Drill with metadata files
> created from another. We can worry about the issue later only if a user
> actually runs into it.
>
> As suggested earlier, let’s just document this obscure case in a release
> note.
>
> - Paul
>
> > On Jul 12, 2017, at 3:04 PM, Jinfeng Ni <jni@apache.org> wrote:
> >
> > Upgrade from 1.10 to 1.11 is irreversible only when parquet metadata
> cache
> > file is involved.
> >
> > I might be wrong, but I thought that if someone is running on 1.10.0, and
> > he wants to go back to 1.0.0, where the first version of parquet
> meatadata
> > cache was introduced, can we guarantee that Drillbit would work
> correctly?
> > I'm trying to understand what DRILL-5660 asks for is a new requirement,
> or
> > an requirement being there starting from Drill 1.0.0?
> >
> > Regression failures are normally regarded as release blocker.
> >
> >
> >
> > On Wed, Jul 12, 2017 at 2:54 PM, Paul Rogers <progers@mapr.com> wrote:
> >
> >> If we don’t do DRILL-5660 for 1.11 then we don’t need to do it at all,
> so
> >> that is a time savings. We could handle this via a release note item
> that
> >> simply states that an upgrade from 1.10 to 1.11 is irreversible: we do
> not
> >> support the ability to fall back to the old version. Everyone OK with
> that?
> >>
> >> We are seeing some stress test regression failures that, I’m told, will
> >> show up as JIRA entries shortly.
> >>
> >> - Paul
> >>
> >>> On Jul 12, 2017, at 11:05 AM, Jinfeng Ni <jni@apache.org> wrote:
> >>>
> >>> I put some analysis in DRILL-5468.
> >>>
> >>> Also, IMHO, DRILL-5660 is not a release blocker.
> >>>
> >>>
> >>>
> >>> On Tue, Jul 11, 2017 at 11:27 AM, Parth Chandra <parthc@apache.org>
> >> wrote:
> >>>
> >>>> Re DRILL-5634:
> >>>> From the Apache commons web page [1] -
> >>>>
> >>>> Please note that Apache Commons Crypto doesn't implement the
> >> cryptographic
> >>>> algorithm such as AES directly.* It wraps to Openssl or JCE which
> >> implement
> >>>> the algorithms.*
> >>>>
> >>>>
> >>>> Since we are cleared for using Openssl and JCE, we are OK to include
> the
> >>>> AES code in the UDFs.
> >>>>
> >>>>
> >>>> Parth
> >>>>
> >>>> [1] https://commons.apache.org/proper/commons-crypto/
> >>>>
> >>>> On Tue, Jul 11, 2017 at 10:56 AM, Arina Yelchiyeva <
> >>>> arina.yelchiyeva@gmail.com> wrote:
> >>>>
> >>>>> During today hangouts we agreed on approximate cut-off date - 14
> July,
> >>>> 2017
> >>>>> (date may be shifted due to the blockers).
> >>>>> I'll start preparing test RC on Thursday to solve any release
> >> preparation
> >>>>> issues beforehand.
> >>>>>
> >>>>> *Blockers*
> >>>>> DRILL-5660 - Vitalii
> >>>>>
> >>>>> *Possible blockers*
> >>>>> DRILL-5468 - Jinfeng
> >>>>> NPE on CSV source - Paul
> >>>>> DRILL-5659 - Parth
> >>>>>
> >>>>> *Will be included if done before the cut-off date*
> >>>>> DRILL-5616 / DRILL-5665 - Boaz
> >>>>> DRILL-5634 - Parth will take a look at Apache commons implementation
> >> and
> >>>>> update.
> >>>>>
> >>>>> Kind regards
> >>>>> Arina
> >>>>>
> >>>>> On Tue, Jul 11, 2017 at 3:14 PM, Charles Givre <cgivre@gmail.com>
> >> wrote:
> >>>>>
> >>>>>> Hi Arina,
> >>>>>> I can’t make the hangout, but we have a few options.  Basically,
> since
> >>>>> all
> >>>>>> the hashing algorithms have no export restrictions, we could:
> >>>>>> 1.  Remove the AES encrypt/decrypt until I can figure out what
we
> have
> >>>> to
> >>>>>> do (and do it)
> >>>>>> 2.  I can try to research it today or tomorrow to figure out
exactly
> >>>> what
> >>>>>> we have to do to include AES.
> >>>>>> 3.  Remove the entire package
> >>>>>>
> >>>>>> I’d like to see the hashing functions at least included because
> we’ve
> >>>> had
> >>>>>> a lot of requests for that functionality.   I also don’t want
this
> to
> >>>>> hold
> >>>>>> things up, so please let me know your preference.  Regardless,
I’m
> >>>> going
> >>>>> to
> >>>>>> start trying to figure out the restrictions so we’ll know
for the
> >>>> future.
> >>>>>>
> >>>>>> Best,
> >>>>>> — C
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> On Jul 11, 2017, at 05:58, Arina Yelchiyeva <
> >>>>> arina.yelchiyeva@gmail.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> @Jinfeng,
> >>>>>>> Let's discuss today on Drill hangout final cut-off date.
> >>>>>>>
> >>>>>>> @Charles
> >>>>>>> Since there were questions about AES in DRILL-5634, should
we
> exclude
> >>>>> it
> >>>>>>> from the candidates for 1.11.0 release?
> >>>>>>>
> >>>>>>> @Boaz
> >>>>>>> Do we have any ETA when DRILL-5616 and DRILL-5665 will pass
code
> >>>>> review?
> >>>>>>>
> >>>>>>> Also I believe we should include DRILL-5660 where we need
to bump
> up
> >>>>>>> parquet metadata version.
> >>>>>>> @Vitalii
> >>>>>>> Do you have any ETA for the fix?
> >>>>>>>
> >>>>>>> Kind regards
> >>>>>>> Arina
> >>>>>>>
> >>>>>>> On Tue, Jul 11, 2017 at 6:02 AM, Boaz Ben-Zvi <bben-zvi@mapr.com>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Arina,
> >>>>>>>>
> >>>>>>>>   Two PR have just been submitted, relating to the Hash
Aggr
> >>>> Spill:
> >>>>>>>> DRILL-5616 and DRILL-5665 .
> >>>>>>>> Can these be considered for 1.11 as well ?
> >>>>>>>>
> >>>>>>>>   Thanks,
> >>>>>>>>
> >>>>>>>>             Boaz
> >>>>>>>>
> >>>>>>>> On 7/10/17, 1:49 PM, "Paul Rogers" <progers@mapr.com>
wrote:
> >>>>>>>>
> >>>>>>>>  DRILL-5601 is no longer a blocker for 1.11; we’ll
defer that
> >>>> until
> >>>>>>>> after the release to allow others more time to work
on other
> >>>> features
> >>>>>>>> rather than doing code reviews.
> >>>>>>>>
> >>>>>>>>  - Paul
> >>>>>>>>
> >>>>>>>>> On Jul 10, 2017, at 1:43 PM, Jinfeng Ni <jni@apache.org>
wrote:
> >>>>>>>>>
> >>>>>>>>> Hi Arina,
> >>>>>>>>>
> >>>>>>>>> It has been a while since we discussed 1.11.0 release.
 Any
> update
> >>>>>>>> on the
> >>>>>>>>> new release plan?  Do we have a cut-off date ?
> >>>>>>>>>
> >>>>>>>>> In the past, I believe Drill release is a time-bounded,
not
> feature
> >>>>>>>>> bounded. If certain fixes/features could not make
it within a
> >>>> certain
> >>>>>>>>> time-frame, we had better defer them into future
release, unless
> >>>>>>>> those
> >>>>>>>>> fixes/features are regarded as blocking issues for
a new release.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Jul 4, 2017 at 5:14 AM, Arina Yelchiyeva
<
> >>>>>>>> arina.yelchiyeva@gmail.com
> >>>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Latest update:
> >>>>>>>>>>
> >>>>>>>>>> We wait for the following Jiras before final
cut off.
> >>>>>>>>>> 1. DRILL-5601  - Paul
> >>>>>>>>>> 2. DRILL-5420 - Kunal
> >>>>>>>>>> 3. DRILL-5634 - Charles
> >>>>>>>>>>
> >>>>>>>>>> Kind regards
> >>>>>>>>>> Arina
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Jun 30, 2017 at 9:29 PM, Kunal Khatua
<kkhatua@mapr.com
> >
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Parth's already +1'd the process with a
minor update. So, we're
> >>>>>>>> good to
> >>>>>>>>>> go
> >>>>>>>>>>> on DRILL-5420. I'll change the label on
that to
> ready-to-commit.
> >>>>>>>>>>>
> >>>>>>>>>>> <http://www.mapr.com/>
> >>>>>>>>>>>
> >>>>>>>>>>> ________________________________
> >>>>>>>>>>> From: Parth Chandra <parthc@apache.org>
> >>>>>>>>>>> Sent: Thursday, June 29, 2017 9:44:39 AM
> >>>>>>>>>>> To: dev@drill.apache.org
> >>>>>>>>>>> Subject: Re: [DISCUSS] Drill 1.11.0 release
> >>>>>>>>>>>
> >>>>>>>>>>> I'm Ok with the pcap format plugin if Pauls
comments are also
> >>>>>>>> addressed.
> >>>>>>>>>> If
> >>>>>>>>>>> the changes are done before your cutoff
deadline,we can merge
> it
> >>>>>>>> in.
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Jun 29, 2017 at 12:26 AM, Arina
Yelchiyeva <
> >>>>>>>>>>> arina.yelchiyeva@gmail.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi all,
> >>>>>>>>>>>>
> >>>>>>>>>>>> here is the list of Jira we are waiting
before making final
> cut
> >>>>>>>> off:
> >>>>>>>>>>>> 1. DRILL-5601  - Paul
> >>>>>>>>>>>> 2. DRILL-3640  - Kunal
> >>>>>>>>>>>> 3. DRILL-5420 - Kunal
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please let me know if there are any
others Jiras we should
> wait
> >>>>>>>> for.
> >>>>>>>>>>>>
> >>>>>>>>>>>> @Charles,
> >>>>>>>>>>>> you have mentioned that you want to
include some UDFs in Drill
> >>>>>>>> 1.11, do
> >>>>>>>>>>> you
> >>>>>>>>>>>> have any ETA for the PR?
> >>>>>>>>>>>>
> >>>>>>>>>>>> @Paul / Parth
> >>>>>>>>>>>> I see there is active CR for pcap format
PR, do we want to
> >>>>>>>> include it
> >>>>>>>>>> in
> >>>>>>>>>>>> this release as well?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Kind regards
> >>>>>>>>>>>> Arina
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Jun 29, 2017 at 10:05 AM, Arina
Yelchiyeva <
> >>>>>>>>>>>> arina.yelchiyeva@gmail.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Parth, thanks a lot the instructions.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Jun 28, 2017 at 8:24 PM,
Parth Chandra <
> >>>>>>>> parthc@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> FWIW, I put release instructions
culled from gists written
> by
> >>>>>>>>>> previous
> >>>>>>>>>>>>>> release managers into an updated
gist here [1]
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> HTH
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Parth
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [1] https://github.com/parthchandra/drill/wiki/Drill-
> >>>>>>>>>> Release-Process
> >>>>>>>>>>>>>> <https://github.com/parthchandra/drill/wiki/Drill-
> >>>>>>>> Release-Process>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Jun 27, 2017 at 5:24
PM, Paul Rogers <
> >>>> progers@mapr.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi Arina,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I have two projects in progress:
one is a candidate for
> 1.11,
> >>>>>>>> the
> >>>>>>>>>>>> other
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>>> not.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The “managed external
sort” is the good candidate for 1.11:
> >>>>>>>> The PR
> >>>>>>>>>>> for
> >>>>>>>>>>>>>>> DRILL-5601 is being reviewed.
Getting that in will allow us
> >>>> to
> >>>>>>>>>>> enable
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> “new” external sort
by default.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The other project is to
address Drill’s memory
> fragmentation
> >>>>>>>> issue
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>>> vectors over 16 MB in size.
Two PRs have been open for a
> >>>> while.
> >>>>>>>>>> They
> >>>>>>>>>>>> are
> >>>>>>>>>>>>>>> the first of several. Each
takes a while to review. So, the
> >>>>>>>>>> project
> >>>>>>>>>>>> does
> >>>>>>>>>>>>>>> not fit 1.11. The work can
go into the next release
> instead.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> - Paul
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message