drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arina Yelchiyeva <arina.yelchiy...@gmail.com>
Subject Re: [DISCUSS] Drill 1.11.0 release
Date Tue, 18 Jul 2017 10:04:53 GMT
Hi Parth,

no problem, I did not start preparing the release since there is one
blocker has left:

*DRILL-5669 - Boaz Ben-Zvi (fix will be ready in a couple of days)*


Kind regards
Arina

On Mon, Jul 17, 2017 at 11:26 PM, Parth Chandra <parthc@apache.org> wrote:

> Hi Arina,
>
>   I just realized that I did not push the C++ client fix ( DRILL-5659) to
> master. I have just done so.
>
>   Sorry if I messed things up for the release, but we should get this fix
> in.
>
> Parth
>
> On Fri, Jul 14, 2017 at 8:15 AM, Paul Rogers <progers@mapr.com> wrote:
>
> > I looked into one NPE on CSV. Turns out it is already fixed by previous
> > 1.11 PRs, so we’re good.
> >
> > There is another reported by a user on the user list, but I’ve not heard
> > back about getting sample data. Let’s not hold the release for that.
> >
> > - Paul
> >
> > > On Jul 14, 2017, at 2:38 AM, Arina Yelchiyeva <
> > arina.yelchiyeva@gmail.com> wrote:
> > >
> > > *Blockers*
> > > DRILL-5659 - Parth (done)
> > > DRILL-5669 - Boaz Ben-Zvi (fix will be ready in a couple of days)
> > >
> > > *Possible blockers*
> > > NPE on CSV - Paul can you please provide an update?
> > >
> > > Kind regards
> > > Arina
> > >
> > > On Thu, Jul 13, 2017 at 2:46 PM, Arina Yelchiyeva <
> > > arina.yelchiyeva@gmail.com> wrote:
> > >
> > >> *Blockers*
> > >> DRILL-5659 - problem is identified, Parth is working on the fix.
> > >>
> > >> *Not blockers*
> > >> DRILL-5660 - Vitalii
> > >> DRILL-5468 - Jinfeng
> > >>
> > >> *Possible blockers*
> > >> DRILL-5669 - Boaz Ben-Zvi / Paul
> > >> NPE on CSV - Paul
> > >>
> > >> Kind regards
> > >> Arina
> > >>
> > >>
> > >> On Thu, Jul 13, 2017 at 1:28 AM, Paul Rogers <progers@mapr.com>
> wrote:
> > >>
> > >>> I withdraw my suggestion to fix DRILL-5660. Was a big hassle for us
> to
> > >>> track down the incompatibility the issue internally, but production
> > users
> > >>> probably don’t attempt to use one build of Drill with metadata files
> > >>> created from another. We can worry about the issue later only if a
> user
> > >>> actually runs into it.
> > >>>
> > >>> As suggested earlier, let’s just document this obscure case in a
> > release
> > >>> note.
> > >>>
> > >>> - Paul
> > >>>
> > >>>> On Jul 12, 2017, at 3:04 PM, Jinfeng Ni <jni@apache.org>
wrote:
> > >>>>
> > >>>> Upgrade from 1.10 to 1.11 is irreversible only when parquet metadata
> > >>> cache
> > >>>> file is involved.
> > >>>>
> > >>>> I might be wrong, but I thought that if someone is running on
> 1.10.0,
> > >>> and
> > >>>> he wants to go back to 1.0.0, where the first version of parquet
> > >>> meatadata
> > >>>> cache was introduced, can we guarantee that Drillbit would work
> > >>> correctly?
> > >>>> I'm trying to understand what DRILL-5660 asks for is a new
> > requirement,
> > >>> or
> > >>>> an requirement being there starting from Drill 1.0.0?
> > >>>>
> > >>>> Regression failures are normally regarded as release blocker.
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Wed, Jul 12, 2017 at 2:54 PM, Paul Rogers <progers@mapr.com>
> > wrote:
> > >>>>
> > >>>>> If we don’t do DRILL-5660 for 1.11 then we don’t need to
do it at
> > all,
> > >>> so
> > >>>>> that is a time savings. We could handle this via a release
note
> item
> > >>> that
> > >>>>> simply states that an upgrade from 1.10 to 1.11 is irreversible:
we
> > do
> > >>> not
> > >>>>> support the ability to fall back to the old version. Everyone
OK
> with
> > >>> that?
> > >>>>>
> > >>>>> We are seeing some stress test regression failures that, I’m
told,
> > will
> > >>>>> show up as JIRA entries shortly.
> > >>>>>
> > >>>>> - Paul
> > >>>>>
> > >>>>>> On Jul 12, 2017, at 11:05 AM, Jinfeng Ni <jni@apache.org>
wrote:
> > >>>>>>
> > >>>>>> I put some analysis in DRILL-5468.
> > >>>>>>
> > >>>>>> Also, IMHO, DRILL-5660 is not a release blocker.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Tue, Jul 11, 2017 at 11:27 AM, Parth Chandra <
> parthc@apache.org>
> > >>>>> wrote:
> > >>>>>>
> > >>>>>>> Re DRILL-5634:
> > >>>>>>> From the Apache commons web page [1] -
> > >>>>>>>
> > >>>>>>> Please note that Apache Commons Crypto doesn't implement
the
> > >>>>> cryptographic
> > >>>>>>> algorithm such as AES directly.* It wraps to Openssl
or JCE which
> > >>>>> implement
> > >>>>>>> the algorithms.*
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Since we are cleared for using Openssl and JCE, we
are OK to
> > include
> > >>> the
> > >>>>>>> AES code in the UDFs.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Parth
> > >>>>>>>
> > >>>>>>> [1] https://commons.apache.org/proper/commons-crypto/
> > >>>>>>>
> > >>>>>>> On Tue, Jul 11, 2017 at 10:56 AM, Arina Yelchiyeva
<
> > >>>>>>> arina.yelchiyeva@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>>> During today hangouts we agreed on approximate
cut-off date - 14
> > >>> July,
> > >>>>>>> 2017
> > >>>>>>>> (date may be shifted due to the blockers).
> > >>>>>>>> I'll start preparing test RC on Thursday to solve
any release
> > >>>>> preparation
> > >>>>>>>> issues beforehand.
> > >>>>>>>>
> > >>>>>>>> *Blockers*
> > >>>>>>>> DRILL-5660 - Vitalii
> > >>>>>>>>
> > >>>>>>>> *Possible blockers*
> > >>>>>>>> DRILL-5468 - Jinfeng
> > >>>>>>>> NPE on CSV source - Paul
> > >>>>>>>> DRILL-5659 - Parth
> > >>>>>>>>
> > >>>>>>>> *Will be included if done before the cut-off date*
> > >>>>>>>> DRILL-5616 / DRILL-5665 - Boaz
> > >>>>>>>> DRILL-5634 - Parth will take a look at Apache commons
> > implementation
> > >>>>> and
> > >>>>>>>> update.
> > >>>>>>>>
> > >>>>>>>> Kind regards
> > >>>>>>>> Arina
> > >>>>>>>>
> > >>>>>>>> On Tue, Jul 11, 2017 at 3:14 PM, Charles Givre
<
> cgivre@gmail.com>
> > >>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi Arina,
> > >>>>>>>>> I can’t make the hangout, but we have a few
options.
> Basically,
> > >>> since
> > >>>>>>>> all
> > >>>>>>>>> the hashing algorithms have no export restrictions,
we could:
> > >>>>>>>>> 1.  Remove the AES encrypt/decrypt until I
can figure out what
> we
> > >>> have
> > >>>>>>> to
> > >>>>>>>>> do (and do it)
> > >>>>>>>>> 2.  I can try to research it today or tomorrow
to figure out
> > >>> exactly
> > >>>>>>> what
> > >>>>>>>>> we have to do to include AES.
> > >>>>>>>>> 3.  Remove the entire package
> > >>>>>>>>>
> > >>>>>>>>> I’d like to see the hashing functions at
least included because
> > >>> we’ve
> > >>>>>>> had
> > >>>>>>>>> a lot of requests for that functionality. 
 I also don’t want
> > this
> > >>> to
> > >>>>>>>> hold
> > >>>>>>>>> things up, so please let me know your preference.
 Regardless,
> > I’m
> > >>>>>>> going
> > >>>>>>>> to
> > >>>>>>>>> start trying to figure out the restrictions
so we’ll know for
> the
> > >>>>>>> future.
> > >>>>>>>>>
> > >>>>>>>>> Best,
> > >>>>>>>>> — C
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>> On Jul 11, 2017, at 05:58, Arina Yelchiyeva
<
> > >>>>>>>> arina.yelchiyeva@gmail.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> @Jinfeng,
> > >>>>>>>>>> Let's discuss today on Drill hangout final
cut-off date.
> > >>>>>>>>>>
> > >>>>>>>>>> @Charles
> > >>>>>>>>>> Since there were questions about AES in
DRILL-5634, should we
> > >>> exclude
> > >>>>>>>> it
> > >>>>>>>>>> from the candidates for 1.11.0 release?
> > >>>>>>>>>>
> > >>>>>>>>>> @Boaz
> > >>>>>>>>>> Do we have any ETA when DRILL-5616 and
DRILL-5665 will pass
> code
> > >>>>>>>> review?
> > >>>>>>>>>>
> > >>>>>>>>>> Also I believe we should include DRILL-5660
where we need to
> > bump
> > >>> up
> > >>>>>>>>>> parquet metadata version.
> > >>>>>>>>>> @Vitalii
> > >>>>>>>>>> Do you have any ETA for the fix?
> > >>>>>>>>>>
> > >>>>>>>>>> Kind regards
> > >>>>>>>>>> Arina
> > >>>>>>>>>>
> > >>>>>>>>>> On Tue, Jul 11, 2017 at 6:02 AM, Boaz Ben-Zvi
<
> > bben-zvi@mapr.com>
> > >>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Hi Arina,
> > >>>>>>>>>>>
> > >>>>>>>>>>>  Two PR have just been submitted, relating
to the Hash Aggr
> > >>>>>>> Spill:
> > >>>>>>>>>>> DRILL-5616 and DRILL-5665 .
> > >>>>>>>>>>> Can these be considered for 1.11 as
well ?
> > >>>>>>>>>>>
> > >>>>>>>>>>>  Thanks,
> > >>>>>>>>>>>
> > >>>>>>>>>>>            Boaz
> > >>>>>>>>>>>
> > >>>>>>>>>>> On 7/10/17, 1:49 PM, "Paul Rogers"
<progers@mapr.com> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> DRILL-5601 is no longer a blocker for
1.11; we’ll defer that
> > >>>>>>> until
> > >>>>>>>>>>> after the release to allow others more
time to work on other
> > >>>>>>> features
> > >>>>>>>>>>> rather than doing code reviews.
> > >>>>>>>>>>>
> > >>>>>>>>>>> - Paul
> > >>>>>>>>>>>
> > >>>>>>>>>>>> On Jul 10, 2017, at 1:43 PM, Jinfeng
Ni <jni@apache.org>
> > wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Hi Arina,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> It has been a while since we discussed
1.11.0 release.  Any
> > >>> update
> > >>>>>>>>>>> on the
> > >>>>>>>>>>>> new release plan?  Do we have a
cut-off date ?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> In the past, I believe Drill release
is a time-bounded, not
> > >>> feature
> > >>>>>>>>>>>> bounded. If certain fixes/features
could not make it within
> a
> > >>>>>>> certain
> > >>>>>>>>>>>> time-frame, we had better defer
them into future release,
> > unless
> > >>>>>>>>>>> those
> > >>>>>>>>>>>> fixes/features are regarded as
blocking issues for a new
> > >>> release.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Tue, Jul 4, 2017 at 5:14 AM,
Arina Yelchiyeva <
> > >>>>>>>>>>> arina.yelchiyeva@gmail.com
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Latest update:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> We wait for the following Jiras
before final cut off.
> > >>>>>>>>>>>>> 1. DRILL-5601  - Paul
> > >>>>>>>>>>>>> 2. DRILL-5420 - Kunal
> > >>>>>>>>>>>>> 3. DRILL-5634 - Charles
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Kind regards
> > >>>>>>>>>>>>> Arina
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Fri, Jun 30, 2017 at 9:29
PM, Kunal Khatua <
> > >>> kkhatua@mapr.com>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Parth's already +1'd the
process with a minor update. So,
> > >>> we're
> > >>>>>>>>>>> good to
> > >>>>>>>>>>>>> go
> > >>>>>>>>>>>>>> on DRILL-5420. I'll change
the label on that to
> > >>> ready-to-commit.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> <http://www.mapr.com/>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> ________________________________
> > >>>>>>>>>>>>>> From: Parth Chandra <parthc@apache.org>
> > >>>>>>>>>>>>>> Sent: Thursday, June 29,
2017 9:44:39 AM
> > >>>>>>>>>>>>>> To: dev@drill.apache.org
> > >>>>>>>>>>>>>> Subject: Re: [DISCUSS]
Drill 1.11.0 release
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> I'm Ok with the pcap format
plugin if Pauls comments are
> > also
> > >>>>>>>>>>> addressed.
> > >>>>>>>>>>>>> If
> > >>>>>>>>>>>>>> the changes are done before
your cutoff deadline,we can
> > merge
> > >>> it
> > >>>>>>>>>>> in.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Thu, Jun 29, 2017 at
12:26 AM, Arina Yelchiyeva <
> > >>>>>>>>>>>>>> arina.yelchiyeva@gmail.com>
wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Hi all,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> here is the list of
Jira we are waiting before making
> final
> > >>> cut
> > >>>>>>>>>>> off:
> > >>>>>>>>>>>>>>> 1. DRILL-5601  - Paul
> > >>>>>>>>>>>>>>> 2. DRILL-3640  - Kunal
> > >>>>>>>>>>>>>>> 3. DRILL-5420 - Kunal
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Please let me know
if there are any others Jiras we
> should
> > >>> wait
> > >>>>>>>>>>> for.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> @Charles,
> > >>>>>>>>>>>>>>> you have mentioned
that you want to include some UDFs in
> > >>> Drill
> > >>>>>>>>>>> 1.11, do
> > >>>>>>>>>>>>>> you
> > >>>>>>>>>>>>>>> have any ETA for the
PR?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> @Paul / Parth
> > >>>>>>>>>>>>>>> I see there is active
CR for pcap format PR, do we want
> to
> > >>>>>>>>>>> include it
> > >>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>> this release as well?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Kind regards
> > >>>>>>>>>>>>>>> Arina
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Thu, Jun 29, 2017
at 10:05 AM, Arina Yelchiyeva <
> > >>>>>>>>>>>>>>> arina.yelchiyeva@gmail.com>
wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Parth, thanks a
lot the instructions.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Wed, Jun 28,
2017 at 8:24 PM, Parth Chandra <
> > >>>>>>>>>>> parthc@apache.org>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> FWIW, I put
release instructions culled from gists
> > written
> > >>> by
> > >>>>>>>>>>>>> previous
> > >>>>>>>>>>>>>>>>> release managers
into an updated gist here [1]
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> HTH
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Parth
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> [1] https://github.com/parthchandra/drill/wiki/Drill-
> > >>>>>>>>>>>>> Release-Process
> > >>>>>>>>>>>>>>>>> <https://github.com/parthchandra/drill/wiki/Drill-
> > >>>>>>>>>>> Release-Process>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Tue, Jun
27, 2017 at 5:24 PM, Paul Rogers <
> > >>>>>>> progers@mapr.com>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Hi Arina,
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> I have
two projects in progress: one is a candidate
> for
> > >>> 1.11,
> > >>>>>>>>>>> the
> > >>>>>>>>>>>>>>> other
> > >>>>>>>>>>>>>>>>> is
> > >>>>>>>>>>>>>>>>>> not.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> The “managed
external sort” is the good candidate for
> > >>> 1.11:
> > >>>>>>>>>>> The PR
> > >>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>> DRILL-5601
is being reviewed. Getting that in will
> allow
> > >>> us
> > >>>>>>> to
> > >>>>>>>>>>>>>> enable
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> “new”
external sort by default.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> The other
project is to address Drill’s memory
> > >>> fragmentation
> > >>>>>>>>>>> issue
> > >>>>>>>>>>>>>>> with
> > >>>>>>>>>>>>>>>>>> vectors
over 16 MB in size. Two PRs have been open
> for a
> > >>>>>>> while.
> > >>>>>>>>>>>>> They
> > >>>>>>>>>>>>>>> are
> > >>>>>>>>>>>>>>>>>> the first
of several. Each takes a while to review.
> So,
> > >>> the
> > >>>>>>>>>>>>> project
> > >>>>>>>>>>>>>>> does
> > >>>>>>>>>>>>>>>>>> not fit
1.11. The work can go into the next release
> > >>> instead.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> - Paul
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>>>
> > >>>
> > >>>
> > >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message