drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arina Yelchiyeva <arina.yelchiy...@gmail.com>
Subject Re: [DISCUSS] Drill 1.11.0 release
Date Fri, 14 Jul 2017 09:38:18 GMT
*Blockers*
DRILL-5659 - Parth (done)
DRILL-5669 - Boaz Ben-Zvi (fix will be ready in a couple of days)

*Possible blockers*
NPE on CSV - Paul can you please provide an update?

Kind regards
Arina

On Thu, Jul 13, 2017 at 2:46 PM, Arina Yelchiyeva <
arina.yelchiyeva@gmail.com> wrote:

> *Blockers*
> DRILL-5659 - problem is identified, Parth is working on the fix.
>
> *Not blockers*
> DRILL-5660 - Vitalii
> DRILL-5468 - Jinfeng
>
> *Possible blockers*
> DRILL-5669 - Boaz Ben-Zvi / Paul
> NPE on CSV - Paul
>
> Kind regards
> Arina
>
>
> On Thu, Jul 13, 2017 at 1:28 AM, Paul Rogers <progers@mapr.com> wrote:
>
>> I withdraw my suggestion to fix DRILL-5660. Was a big hassle for us to
>> track down the incompatibility the issue internally, but production users
>> probably don’t attempt to use one build of Drill with metadata files
>> created from another. We can worry about the issue later only if a user
>> actually runs into it.
>>
>> As suggested earlier, let’s just document this obscure case in a release
>> note.
>>
>> - Paul
>>
>> > On Jul 12, 2017, at 3:04 PM, Jinfeng Ni <jni@apache.org> wrote:
>> >
>> > Upgrade from 1.10 to 1.11 is irreversible only when parquet metadata
>> cache
>> > file is involved.
>> >
>> > I might be wrong, but I thought that if someone is running on 1.10.0,
>> and
>> > he wants to go back to 1.0.0, where the first version of parquet
>> meatadata
>> > cache was introduced, can we guarantee that Drillbit would work
>> correctly?
>> > I'm trying to understand what DRILL-5660 asks for is a new requirement,
>> or
>> > an requirement being there starting from Drill 1.0.0?
>> >
>> > Regression failures are normally regarded as release blocker.
>> >
>> >
>> >
>> > On Wed, Jul 12, 2017 at 2:54 PM, Paul Rogers <progers@mapr.com> wrote:
>> >
>> >> If we don’t do DRILL-5660 for 1.11 then we don’t need to do it at all,
>> so
>> >> that is a time savings. We could handle this via a release note item
>> that
>> >> simply states that an upgrade from 1.10 to 1.11 is irreversible: we do
>> not
>> >> support the ability to fall back to the old version. Everyone OK with
>> that?
>> >>
>> >> We are seeing some stress test regression failures that, I’m told, will
>> >> show up as JIRA entries shortly.
>> >>
>> >> - Paul
>> >>
>> >>> On Jul 12, 2017, at 11:05 AM, Jinfeng Ni <jni@apache.org> wrote:
>> >>>
>> >>> I put some analysis in DRILL-5468.
>> >>>
>> >>> Also, IMHO, DRILL-5660 is not a release blocker.
>> >>>
>> >>>
>> >>>
>> >>> On Tue, Jul 11, 2017 at 11:27 AM, Parth Chandra <parthc@apache.org>
>> >> wrote:
>> >>>
>> >>>> Re DRILL-5634:
>> >>>> From the Apache commons web page [1] -
>> >>>>
>> >>>> Please note that Apache Commons Crypto doesn't implement the
>> >> cryptographic
>> >>>> algorithm such as AES directly.* It wraps to Openssl or JCE which
>> >> implement
>> >>>> the algorithms.*
>> >>>>
>> >>>>
>> >>>> Since we are cleared for using Openssl and JCE, we are OK to include
>> the
>> >>>> AES code in the UDFs.
>> >>>>
>> >>>>
>> >>>> Parth
>> >>>>
>> >>>> [1] https://commons.apache.org/proper/commons-crypto/
>> >>>>
>> >>>> On Tue, Jul 11, 2017 at 10:56 AM, Arina Yelchiyeva <
>> >>>> arina.yelchiyeva@gmail.com> wrote:
>> >>>>
>> >>>>> During today hangouts we agreed on approximate cut-off date
- 14
>> July,
>> >>>> 2017
>> >>>>> (date may be shifted due to the blockers).
>> >>>>> I'll start preparing test RC on Thursday to solve any release
>> >> preparation
>> >>>>> issues beforehand.
>> >>>>>
>> >>>>> *Blockers*
>> >>>>> DRILL-5660 - Vitalii
>> >>>>>
>> >>>>> *Possible blockers*
>> >>>>> DRILL-5468 - Jinfeng
>> >>>>> NPE on CSV source - Paul
>> >>>>> DRILL-5659 - Parth
>> >>>>>
>> >>>>> *Will be included if done before the cut-off date*
>> >>>>> DRILL-5616 / DRILL-5665 - Boaz
>> >>>>> DRILL-5634 - Parth will take a look at Apache commons implementation
>> >> and
>> >>>>> update.
>> >>>>>
>> >>>>> Kind regards
>> >>>>> Arina
>> >>>>>
>> >>>>> On Tue, Jul 11, 2017 at 3:14 PM, Charles Givre <cgivre@gmail.com>
>> >> wrote:
>> >>>>>
>> >>>>>> Hi Arina,
>> >>>>>> I can’t make the hangout, but we have a few options. 
Basically,
>> since
>> >>>>> all
>> >>>>>> the hashing algorithms have no export restrictions, we could:
>> >>>>>> 1.  Remove the AES encrypt/decrypt until I can figure out
what we
>> have
>> >>>> to
>> >>>>>> do (and do it)
>> >>>>>> 2.  I can try to research it today or tomorrow to figure
out
>> exactly
>> >>>> what
>> >>>>>> we have to do to include AES.
>> >>>>>> 3.  Remove the entire package
>> >>>>>>
>> >>>>>> I’d like to see the hashing functions at least included
because
>> we’ve
>> >>>> had
>> >>>>>> a lot of requests for that functionality.   I also don’t
want this
>> to
>> >>>>> hold
>> >>>>>> things up, so please let me know your preference.  Regardless,
I’m
>> >>>> going
>> >>>>> to
>> >>>>>> start trying to figure out the restrictions so we’ll know
for the
>> >>>> future.
>> >>>>>>
>> >>>>>> Best,
>> >>>>>> — C
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>> On Jul 11, 2017, at 05:58, Arina Yelchiyeva <
>> >>>>> arina.yelchiyeva@gmail.com>
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>> @Jinfeng,
>> >>>>>>> Let's discuss today on Drill hangout final cut-off date.
>> >>>>>>>
>> >>>>>>> @Charles
>> >>>>>>> Since there were questions about AES in DRILL-5634,
should we
>> exclude
>> >>>>> it
>> >>>>>>> from the candidates for 1.11.0 release?
>> >>>>>>>
>> >>>>>>> @Boaz
>> >>>>>>> Do we have any ETA when DRILL-5616 and DRILL-5665 will
pass code
>> >>>>> review?
>> >>>>>>>
>> >>>>>>> Also I believe we should include DRILL-5660 where we
need to bump
>> up
>> >>>>>>> parquet metadata version.
>> >>>>>>> @Vitalii
>> >>>>>>> Do you have any ETA for the fix?
>> >>>>>>>
>> >>>>>>> Kind regards
>> >>>>>>> Arina
>> >>>>>>>
>> >>>>>>> On Tue, Jul 11, 2017 at 6:02 AM, Boaz Ben-Zvi <bben-zvi@mapr.com>
>> >>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Hi Arina,
>> >>>>>>>>
>> >>>>>>>>   Two PR have just been submitted, relating to the
Hash Aggr
>> >>>> Spill:
>> >>>>>>>> DRILL-5616 and DRILL-5665 .
>> >>>>>>>> Can these be considered for 1.11 as well ?
>> >>>>>>>>
>> >>>>>>>>   Thanks,
>> >>>>>>>>
>> >>>>>>>>             Boaz
>> >>>>>>>>
>> >>>>>>>> On 7/10/17, 1:49 PM, "Paul Rogers" <progers@mapr.com>
wrote:
>> >>>>>>>>
>> >>>>>>>>  DRILL-5601 is no longer a blocker for 1.11; we’ll
defer that
>> >>>> until
>> >>>>>>>> after the release to allow others more time to work
on other
>> >>>> features
>> >>>>>>>> rather than doing code reviews.
>> >>>>>>>>
>> >>>>>>>>  - Paul
>> >>>>>>>>
>> >>>>>>>>> On Jul 10, 2017, at 1:43 PM, Jinfeng Ni <jni@apache.org>
wrote:
>> >>>>>>>>>
>> >>>>>>>>> Hi Arina,
>> >>>>>>>>>
>> >>>>>>>>> It has been a while since we discussed 1.11.0
release.  Any
>> update
>> >>>>>>>> on the
>> >>>>>>>>> new release plan?  Do we have a cut-off date
?
>> >>>>>>>>>
>> >>>>>>>>> In the past, I believe Drill release is a time-bounded,
not
>> feature
>> >>>>>>>>> bounded. If certain fixes/features could not
make it within a
>> >>>> certain
>> >>>>>>>>> time-frame, we had better defer them into future
release, unless
>> >>>>>>>> those
>> >>>>>>>>> fixes/features are regarded as blocking issues
for a new
>> release.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Tue, Jul 4, 2017 at 5:14 AM, Arina Yelchiyeva
<
>> >>>>>>>> arina.yelchiyeva@gmail.com
>> >>>>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Latest update:
>> >>>>>>>>>>
>> >>>>>>>>>> We wait for the following Jiras before final
cut off.
>> >>>>>>>>>> 1. DRILL-5601  - Paul
>> >>>>>>>>>> 2. DRILL-5420 - Kunal
>> >>>>>>>>>> 3. DRILL-5634 - Charles
>> >>>>>>>>>>
>> >>>>>>>>>> Kind regards
>> >>>>>>>>>> Arina
>> >>>>>>>>>>
>> >>>>>>>>>> On Fri, Jun 30, 2017 at 9:29 PM, Kunal Khatua
<
>> kkhatua@mapr.com>
>> >>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Parth's already +1'd the process with
a minor update. So,
>> we're
>> >>>>>>>> good to
>> >>>>>>>>>> go
>> >>>>>>>>>>> on DRILL-5420. I'll change the label
on that to
>> ready-to-commit.
>> >>>>>>>>>>>
>> >>>>>>>>>>> <http://www.mapr.com/>
>> >>>>>>>>>>>
>> >>>>>>>>>>> ________________________________
>> >>>>>>>>>>> From: Parth Chandra <parthc@apache.org>
>> >>>>>>>>>>> Sent: Thursday, June 29, 2017 9:44:39
AM
>> >>>>>>>>>>> To: dev@drill.apache.org
>> >>>>>>>>>>> Subject: Re: [DISCUSS] Drill 1.11.0
release
>> >>>>>>>>>>>
>> >>>>>>>>>>> I'm Ok with the pcap format plugin if
Pauls comments are also
>> >>>>>>>> addressed.
>> >>>>>>>>>> If
>> >>>>>>>>>>> the changes are done before your cutoff
deadline,we can merge
>> it
>> >>>>>>>> in.
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Thu, Jun 29, 2017 at 12:26 AM, Arina
Yelchiyeva <
>> >>>>>>>>>>> arina.yelchiyeva@gmail.com> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Hi all,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> here is the list of Jira we are
waiting before making final
>> cut
>> >>>>>>>> off:
>> >>>>>>>>>>>> 1. DRILL-5601  - Paul
>> >>>>>>>>>>>> 2. DRILL-3640  - Kunal
>> >>>>>>>>>>>> 3. DRILL-5420 - Kunal
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Please let me know if there are
any others Jiras we should
>> wait
>> >>>>>>>> for.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> @Charles,
>> >>>>>>>>>>>> you have mentioned that you want
to include some UDFs in
>> Drill
>> >>>>>>>> 1.11, do
>> >>>>>>>>>>> you
>> >>>>>>>>>>>> have any ETA for the PR?
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> @Paul / Parth
>> >>>>>>>>>>>> I see there is active CR for pcap
format PR, do we want to
>> >>>>>>>> include it
>> >>>>>>>>>> in
>> >>>>>>>>>>>> this release as well?
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Kind regards
>> >>>>>>>>>>>> Arina
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Thu, Jun 29, 2017 at 10:05 AM,
Arina Yelchiyeva <
>> >>>>>>>>>>>> arina.yelchiyeva@gmail.com> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Parth, thanks a lot the instructions.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Wed, Jun 28, 2017 at 8:24
PM, Parth Chandra <
>> >>>>>>>> parthc@apache.org>
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> FWIW, I put release instructions
culled from gists written
>> by
>> >>>>>>>>>> previous
>> >>>>>>>>>>>>>> release managers into an
updated gist here [1]
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> HTH
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Parth
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> [1] https://github.com/parthchandra/drill/wiki/Drill-
>> >>>>>>>>>> Release-Process
>> >>>>>>>>>>>>>> <https://github.com/parthchandra/drill/wiki/Drill-
>> >>>>>>>> Release-Process>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Tue, Jun 27, 2017 at
5:24 PM, Paul Rogers <
>> >>>> progers@mapr.com>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Hi Arina,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I have two projects
in progress: one is a candidate for
>> 1.11,
>> >>>>>>>> the
>> >>>>>>>>>>>> other
>> >>>>>>>>>>>>>> is
>> >>>>>>>>>>>>>>> not.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> The “managed external
sort” is the good candidate for
>> 1.11:
>> >>>>>>>> The PR
>> >>>>>>>>>>> for
>> >>>>>>>>>>>>>>> DRILL-5601 is being
reviewed. Getting that in will allow
>> us
>> >>>> to
>> >>>>>>>>>>> enable
>> >>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>> “new” external sort
by default.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> The other project is
to address Drill’s memory
>> fragmentation
>> >>>>>>>> issue
>> >>>>>>>>>>>> with
>> >>>>>>>>>>>>>>> vectors over 16 MB in
size. Two PRs have been open for a
>> >>>> while.
>> >>>>>>>>>> They
>> >>>>>>>>>>>> are
>> >>>>>>>>>>>>>>> the first of several.
Each takes a while to review. So,
>> the
>> >>>>>>>>>> project
>> >>>>>>>>>>>> does
>> >>>>>>>>>>>>>>> not fit 1.11. The work
can go into the next release
>> instead.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> - Paul
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>
>> >>
>> >>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message