nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy LoPresto <alopre...@apache.org>
Subject Re: [DISCUSS] NiFi 1.1.0 release
Date Thu, 17 Nov 2016 03:38:53 GMT
Just updating this thread that NIFI-3050 [1] and NIFI-3051 [2] have been added to my plate
for this release. Coordinated with Joe Witt and they should both be included.

[1] https://issues.apache.org/jira/browse/NIFI-3050
[2] https://issues.apache.org/jira/browse/NIFI-3051

Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Nov 16, 2016, at 12:08 PM, Joe Witt <joe.witt@gmail.com> wrote:
> 
> Team
> 
> There is a thread on apache legal-discuss that might allow for a
> graceperiod of continued usage of the json library.  Am going to keep
> a close eye on this and if VP Legal approves we'll be able to keep the
> twitter processors in which is definitely a good thing.  Will advise
> 
> Thanks
> Joe
> 
> On Wed, Nov 16, 2016 at 10:37 AM, Bryan Bende <bbende@gmail.com> wrote:
>> I've noticed an issue with the per-instance class loading capability
>> introduced in NIFI-2909 where the additional classpath resources can get
>> incorrectly removed from the class loader.
>> 
>> I was able to reproduced this with a unit test and have a fix ready. I
>> believe this is important and needs to go in for the 1.1 release, going to
>> re-open NIFI-2909 and submit a PR shortly.
>> 
>> -Bryan
>> 
>> On Wed, Nov 16, 2016 at 8:11 AM, Matt Gilman <matt.c.gilman@gmail.com>
>> wrote:
>> 
>>> I have two items that I would like to wrap up prior to creating an RC for
>>> 1.1.0. NIFI-2949 addresses some UX issues around Remote Process Group port
>>> configuration. The work is already completed and I will be reviewing it
>>> this today. Additionally, following recent interest on the mailing list,
>>> I'd like to knock out NIFI-3020. This will allow an admin to configure a
>>> strategy for user identity when logging in via LDAP. Specifically, it will
>>> support usage of the DN (the default and current implementation) as well as
>>> the username the user logged in as. I should be able to have a PR up for
>>> this work later today.
>>> 
>>> Thanks!
>>> 
>>> Matt
>>> 
>>> [1] https://issues.apache.org/jira/browse/NIFI-2949
>>> [2] https://issues.apache.org/jira/browse/NIFI-3020
>>> 
>>> 
>>> On Tue, Nov 15, 2016 at 8:00 PM, Joe Witt <joe.witt@gmail.com> wrote:
>>> 
>>>> The code is within the twitter4j library itself.  I filed a request to
>>>> twitter4jg.  The most likely case is we will need to submit a PR to them.
>>>> However, I don't see this as something that should delay the release.  We
>>>> can provide instructions for folks wanting to use the processor during
>>> the
>>>> time we cannot make it available in a convenient manner.  I will provide
>>> a
>>>> meaningful comment about this in release notes and pointers on what folks
>>>> can do in the meantime.
>>>> 
>>>> On Nov 15, 2016 7:41 PM, "Andy LoPresto" <alopresto@apache.org> wrote:
>>>> 
>>>>> I understand there was a discussion thread within the NiFi community
>>> for
>>>>> this as well and I missed responding to that at that time. It just
>>> seems
>>>> to
>>>>> me like JSON processing is necessary for GetTwitter, which is
>>> incredibly
>>>>> useful for demonstrating NiFi’s ability to read from a high volume
>>> stream
>>>>> out of the box. With NIFI-3019 (Remove GetTwitter from default build),
>>> is
>>>>> there any related effort to substitute an acceptable replacement JSON
>>>>> library to restore this functionality?
>>>>> 
>>>>> [1] https://issues.apache.org/jira/browse/NIFI-3019
>>>>> 
>>>>> Andy LoPresto
>>>>> alopresto@apache.org
>>>>> *alopresto.apache@gmail.com <alopresto.apache@gmail.com>*
>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>> 
>>>>> On Nov 15, 2016, at 4:36 PM, Andy LoPresto <alopresto@apache.org>
>>> wrote:
>>>>> 
>>>>> I’m working with Bryan Rosander to close out NIFI-3024, NIFI-2655,
and
>>>>> NIFI-2653. I believe Matt Burgess is working on NIFI-3011 and we
>>>>> investigated some alternate TLS config options for the new version of
>>> the
>>>>> client library.
>>>>> 
>>>>> Is there any alternative to excluding the GetTwitter processor? Using
>>>>> Johnzon [1] or the Android re-implementation [2] discussed in the
>>> mailing
>>>>> list thread?
>>>>> 
>>>>> [1] https://johnzon.apache.org/
>>>>> [2] https://developer.android.com/reference/org/json/package-
>>>> summary.html
>>>>> 
>>>>> 
>>>>> Andy LoPresto
>>>>> alopresto@apache.org
>>>>> *alopresto.apache@gmail.com <alopresto.apache@gmail.com>*
>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>> 
>>>>> On Nov 15, 2016, at 3:58 PM, Joe Witt <joe.witt@gmail.com> wrote:
>>>>> 
>>>>> Team
>>>>> 
>>>>> Very happy to see that we are down to three items remaining tagged to
>>>>> 1.1.0.  Solid effort over the recent weeks to close the gap including
>>>> work
>>>>> to get past the now category x Jason dependency we had.  The most
>>> notable
>>>>> impact from that is the wildly popular GetTwitter processor, the fav
>>> new
>>>>> nifi user and demo processor, can no longer be included in the default
>>>>> build.  It is optionally available if users choose to build and use it
>>>> but
>>>>> we won't distribute binaries that have it.
>>>>> 
>>>>> I see some review movement on some patch available but untagged items.
>>>>> 
>>>>> I plan to kick off the 1.1.0 rc work soon. Perhaps Thurs or Fri. Anyone
>>>>> have any outstanding items?
>>>>> 
>>>>> Thanks
>>>>> Joe
>>>>> 
>>>>> On Nov 8, 2016 2:12 PM, "Joe Witt" <joe.witt@gmail.com> wrote:
>>>>> 
>>>>> Ryan
>>>>> 
>>>>> Not officially but I think we should try to close this thing out and
>>>>> start a vote in the next week or two at most.
>>>>> 
>>>>> I'm going through the tickets again now.  There is also a new issue of
>>>>> the json-p license falling out of favor in Apache legal terms and
>>>>> becoming Category-X.  Am looking into that now.
>>>>> 
>>>>> Thanks
>>>>> Joe
>>>>> 
>>>>> On Tue, Nov 8, 2016 at 2:05 PM, Ryan Ward <ryan.ward2@gmail.com>
>>> wrote:
>>>>> 
>>>>> Joe - Is there a target date for 1.1?
>>>>> 
>>>>> On Thu, Oct 27, 2016 at 10:50 AM, Joe Witt <joe.witt@gmail.com>
wrote:
>>>>> 
>>>>> Team,
>>>>> 
>>>>> Just an update on things with working toward an Apache NiFi 1.1.0
>>>>> release.  There are still about 33 JIRAs there now and some are
>>>>> awaiting review and are some are under active progress. Yet there is
>>>>> good traction and progress. I think we should just stay vigilant with
>>>>> what makes it in and keep working it down.  So let's please shoot for
>>>>> a couple weeks from now.  If it is ready sooner I'll jump on it.
>>>>> 
>>>>> Thanks
>>>>> Joe
>>>>> 
>>>>> On Wed, Oct 19, 2016 at 9:06 AM, Joe Witt <joe.witt@gmail.com>
wrote:
>>>>> 
>>>>> Team,
>>>>> 
>>>>> There are 31 open JIRAs at present tagged to Apache NiFi 1.1.0.  Let's
>>>>> avoiding putting more in there for now at least without a discussion.
>>>>> Of the 31 JIRAs there the vast majority need review so we should be
>>>>> able to close these down fairly quickly as long as we don't let the
>>>>> list grow.
>>>>> 
>>>>> Thanks
>>>>> joe
>>>>> 
>>>>> On Fri, Oct 14, 2016 at 4:39 PM, Edgardo Vega <edgardo.vega@gmail.com>
>>>>> 
>>>>> wrote:
>>>>> 
>>>>> Joe,
>>>>> 
>>>>> Appreciate the offer it isn't my PR. I was just using it as an
>>>>> 
>>>>> example.
>>>>> 
>>>>> All
>>>>> 
>>>>> mine are currently closed, which I greatly appreciate.
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Edgardo
>>>>> 
>>>>> On Friday, October 14, 2016, Joe Witt <joe.witt@gmail.com> wrote:
>>>>> 
>>>>> Edgardo,
>>>>> 
>>>>> You mentioned a PR from August. I'd be happy to help you work that
>>>>> through review.
>>>>> 
>>>>> Thanks
>>>>> Joe
>>>>> 
>>>>> On Fri, Oct 14, 2016 at 10:45 AM, Edgardo Vega <
>>>>> 
>>>>> edgardo.vega@gmail.com
>>>>> 
>>>>> <javascript:;>> wrote:
>>>>> 
>>>>> I have agreed that at this point a release is important. My goal
>>>>> 
>>>>> was
>>>>> 
>>>>> try
>>>>> 
>>>>> to
>>>>> 
>>>>> squeeze in a much goodness as possible into the release, but the
>>>>> 
>>>>> important
>>>>> 
>>>>> bug fixes should come first. Getting 1.x into a state where the
>>>>> 
>>>>> release
>>>>> 
>>>>> notes don't say that it is geared toward developers and testers is
>>>>> 
>>>>> really
>>>>> 
>>>>> huge.
>>>>> 
>>>>> I think Nifi is a great community otherwise I would participate in
>>>>> 
>>>>> the
>>>>> 
>>>>> mailing list, create Jira tickets and pull requests. I am only
>>>>> 
>>>>> trying to
>>>>> 
>>>>> strengthen the great thing that is going on here. We can always do
>>>>> 
>>>>> better.
>>>>> 
>>>>> I was not trying to put down this community only to participate and
>>>>> 
>>>>> make
>>>>> 
>>>>> it
>>>>> 
>>>>> better. I think this conversation is an indication of how great
>>>>> 
>>>>> this
>>>>> 
>>>>> community is.
>>>>> 
>>>>> Maybe I am being sensitive about this issue and trying to
>>>>> 
>>>>> strengthen
>>>>> 
>>>>> the
>>>>> 
>>>>> nifi community even more, after coming from a conference where it
>>>>> 
>>>>> was
>>>>> 
>>>>> reported there was lots of excitement at first and now the
>>>>> 
>>>>> participation
>>>>> 
>>>>> in
>>>>> 
>>>>> the community has really died down and they are struggling. I don't
>>>>> 
>>>>> want
>>>>> 
>>>>> to
>>>>> 
>>>>> see that happen here.
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Edgardo
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> On Fri, Oct 14, 2016 at 9:37 AM, Andre <andre-lists@fucs.org
>>>>> 
>>>>> <javascript:;>> wrote:
>>>>> 
>>>>> 
>>>>> Edgardo,
>>>>> 
>>>>> Thank you for your feedback. We hear your comments and as a
>>>>> 
>>>>> committer I
>>>>> 
>>>>> can
>>>>> 
>>>>> share we are constantly looking to improve the PR process, having
>>>>> 
>>>>> already
>>>>> 
>>>>> taken many of the steps you suggest.
>>>>> 
>>>>> However, it is important to notice that the number of PRs should
>>>>> 
>>>>> not be
>>>>> 
>>>>> seen as a metric of engagement by the development community: Most
>>>>> 
>>>>> of us
>>>>> 
>>>>> will submit PRs so that our work can be carefully reviewed by our
>>>>> 
>>>>> peers
>>>>> 
>>>>> and
>>>>> 
>>>>> some of us will use JIRA patches to provide contributions.
>>>>> 
>>>>> Having said that, it is true that some PRs may sit idle for a long
>>>>> 
>>>>> time
>>>>> 
>>>>> and
>>>>> 
>>>>> we are working to improve this pipeline.
>>>>> 
>>>>> It was therefore no coincidence that I  browsed most of the PRs
>>>>> 
>>>>> performing
>>>>> 
>>>>> a triage of items that have been superseded or diverged from the
>>>>> 
>>>>> current
>>>>> 
>>>>> code base.
>>>>> 
>>>>> In fact, less than a month ago the dev team closed a number of
>>>>> 
>>>>> stalled
>>>>> 
>>>>> and
>>>>> 
>>>>> superseded PRs (commit cc5e827aa1dfe2f376e9836380ba63c15269eea8).
>>>>> 
>>>>> Despite all the above, I think Joe has a point. The master
>>>>> 
>>>>> contain a
>>>>> 
>>>>> series
>>>>> 
>>>>> of important bug fixes and suspect the community would benefit
>>>>> 
>>>>> from
>>>>> 
>>>>> a
>>>>> 
>>>>> release sooner rather than later.
>>>>> 
>>>>> Once again, thank you for your feedback and contribution. It is
>>>>> 
>>>>> good to
>>>>> 
>>>>> have you here.
>>>>> 
>>>>> Andre
>>>>> 
>>>>> On Fri, Oct 14, 2016 at 11:30 PM, Edgardo Vega <
>>>>> 
>>>>> edgardo.vega@gmail.com
>>>>> 
>>>>> <javascript:;>>
>>>>> 
>>>>> wrote:
>>>>> 
>>>>> Joe - You are correct I was mentioning the PRs that are
>>>>> 
>>>>> currently
>>>>> 
>>>>> open.
>>>>> 
>>>>> 
>>>>> Regardless of how it happens reducing the count of open PRs I
>>>>> 
>>>>> believe
>>>>> 
>>>>> to
>>>>> 
>>>>> be
>>>>> 
>>>>> extremely important. Maybe I was hoping that the release could
>>>>> 
>>>>> be
>>>>> 
>>>>> a
>>>>> 
>>>>> forcing
>>>>> 
>>>>> function to make that happen. I believe that developers are more
>>>>> 
>>>>> willing
>>>>> 
>>>>> to
>>>>> 
>>>>> contribute when they see that their PRs will actually be able
>>>>> 
>>>>> accepted
>>>>> 
>>>>> and
>>>>> 
>>>>> merged into the code base. Having a low number of open PRs in
>>>>> 
>>>>> progress
>>>>> 
>>>>> is a
>>>>> 
>>>>> great indication that the main nifi developers are fully engaged
>>>>> 
>>>>> with
>>>>> 
>>>>> the
>>>>> 
>>>>> community.
>>>>> 
>>>>> There are a few PRs that don't have any comments from committers
>>>>> 
>>>>> at
>>>>> 
>>>>> all.
>>>>> 
>>>>> I
>>>>> 
>>>>> found one from August in that state. If that was my PR I don't
>>>>> 
>>>>> think I
>>>>> 
>>>>> would be so willing to put another one in anytime soon. I do get
>>>>> 
>>>>> that
>>>>> 
>>>>> sometime PRs get stalled by the originator, if so maybe a rule
>>>>> 
>>>>> about
>>>>> 
>>>>> closing them after a certain amount of time or being taken over
>>>>> 
>>>>> by a
>>>>> 
>>>>> core
>>>>> 
>>>>> contributor if they think it worthwhile.
>>>>> 
>>>>> I would like to shoutout to James Wing on my last PR he was
>>>>> 
>>>>> quick
>>>>> 
>>>>> to
>>>>> 
>>>>> review, provided great comments, testing, and even some
>>>>> 
>>>>> additional
>>>>> 
>>>>> code.
>>>>> 
>>>>> It
>>>>> 
>>>>> was a great PR experience.
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Edgardo
>>>>> 
>>>>> 
>>>>> 
>>>>> On Thu, Oct 13, 2016 at 4:14 PM, Joe Percivall <
>>>>> 
>>>>> joepercivall@yahoo.com <javascript:;>.
>>>>> 
>>>>> invalid> wrote:
>>>>> 
>>>>> Joe, I think you misread. Edgardo is referring to the Pull
>>>>> 
>>>>> Requests
>>>>> 
>>>>> that
>>>>> 
>>>>> are currently open, not the tickets assigned to the 1.1.0
>>>>> 
>>>>> version.
>>>>> 
>>>>> 
>>>>> I think these goals (releasing 1.1.0 and cutting down the PR
>>>>> 
>>>>> count)
>>>>> 
>>>>> should
>>>>> 
>>>>> be two different efforts. Doing a thorough job reviewing
>>>>> 
>>>>> takes a
>>>>> 
>>>>> significant amount of time from both the reviewer and
>>>>> 
>>>>> contributor.
>>>>> 
>>>>> In
>>>>> 
>>>>> order
>>>>> 
>>>>> to cut it down significantly would take much longer than a
>>>>> 
>>>>> couple
>>>>> 
>>>>> days.
>>>>> 
>>>>> 
>>>>> Also there has already been a lot of great new features and
>>>>> 
>>>>> bug
>>>>> 
>>>>> fixes
>>>>> 
>>>>> contributed to the 1.X line and I don't think it's worth
>>>>> 
>>>>> holding up
>>>>> 
>>>>> a
>>>>> 
>>>>> 1.1.0
>>>>> 
>>>>> release for tickets not assigned to this fix version. As an
>>>>> 
>>>>> added
>>>>> 
>>>>> bonus
>>>>> 
>>>>> though, I think many of the tickets tagged as 1.1.0 have PRs
>>>>> 
>>>>> already
>>>>> 
>>>>> open
>>>>> 
>>>>> so closing those will make a large dent in the PR count.
>>>>> 
>>>>> 
>>>>> Joe
>>>>> 
>>>>> - - - - - -
>>>>> Joseph Percivall
>>>>> linkedin.com/in/Percivall
>>>>> e: joepercivall@yahoo.com <javascript:;>
>>>>> 
>>>>> 
>>>>> 
>>>>> On Thursday, October 13, 2016 3:58 PM, Joe Witt <
>>>>> 
>>>>> joe.witt@gmail.com
>>>>> 
>>>>> <javascript:;>>
>>>>> 
>>>>> wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>> There are less than 30 right now.  Many of the roughly 90+
>>>>> 
>>>>> JIRAs
>>>>> 
>>>>> opened on 1.1.0 were easily dispositioned to 1.2.0 or closed
>>>>> 
>>>>> or
>>>>> 
>>>>> just
>>>>> 
>>>>> had fix versions removed.
>>>>> 
>>>>> We will need to have a push over the next bunch of days to
>>>>> 
>>>>> deal
>>>>> 
>>>>> with
>>>>> 
>>>>> reviewing/merging/moving the remaining items.
>>>>> 
>>>>> Thanks
>>>>> Joe
>>>>> 
>>>>> 
>>>>> On Thu, Oct 13, 2016 at 3:49 PM, Edgardo Vega <
>>>>> 
>>>>> edgardo.vega@gmail.com <javascript:;>>
>>>>> 
>>>>> wrote:
>>>>> 
>>>>> Joe,
>>>>> 
>>>>> There are 75 PRs currently open. Why not make a push over
>>>>> 
>>>>> the
>>>>> 
>>>>> next
>>>>> 
>>>>> bunch
>>>>> 
>>>>> of
>>>>> 
>>>>> days to get them closed and then cut the release after that.
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Edgardo
>>>>> 
>>>>> On Thu, Oct 13, 2016 at 12:44 PM, Joe Witt <
>>>>> 
>>>>> joe.witt@gmail.com
>>>>> 
>>>>> <javascript:;>>
>>>>> 
>>>>> wrote:
>>>>> 
>>>>> 
>>>>> Team,
>>>>> 
>>>>> There have been a ton of bugs fixed a few nice features.  I
>>>>> 
>>>>> would
>>>>> 
>>>>> like
>>>>> 
>>>>> to move to get Apache NiFi 1.1.0 release going pretty much
>>>>> 
>>>>> based
>>>>> 
>>>>> on
>>>>> 
>>>>> where we are now and plan to move most tickets to a new
>>>>> 
>>>>> Apache
>>>>> 
>>>>> NiFi
>>>>> 
>>>>> 1.2.0 version.  We can try to get back on our roughly 6-8
>>>>> 
>>>>> week
>>>>> 
>>>>> release
>>>>> 
>>>>> schedule and shoot for a mid to late Nov release for NiFi
>>>>> 
>>>>> 1.2.0
>>>>> 
>>>>> this
>>>>> 
>>>>> way as well. Please advise if anyone has any other views on
>>>>> 
>>>>> this. In
>>>>> 
>>>>> the mean time I'll get the wheels in motion so you'll be
>>>>> 
>>>>> seeing a
>>>>> 
>>>>> lot
>>>>> 
>>>>> of JIRA/issue updates to move version around.
>>>>> 
>>>>> Thanks
>>>>> Joe
>>>>> 
>>>>> On Thu, Oct 13, 2016 at 12:02 PM, Tony Kurc <
>>>>> 
>>>>> trkurc@gmail.com
>>>>> 
>>>>> <javascript:;>>
>>>>> 
>>>>> wrote:
>>>>> 
>>>>> Sounds good Joe. I have no issue to you doing the rm'ing
>>>>> 
>>>>> for
>>>>> 
>>>>> it.
>>>>> 
>>>>> 
>>>>> On Oct 13, 2016 8:19 AM, "Joe Witt" <joe.witt@gmail.com
>>>>> 
>>>>> <javascript:;>> wrote:
>>>>> 
>>>>> 
>>>>> Team,
>>>>> 
>>>>> There are a lot of great fixes and improvements on the
>>>>> 
>>>>> master
>>>>> 
>>>>> line
>>>>> 
>>>>> now
>>>>> 
>>>>> and we're at a good time window to start pushing for a
>>>>> 
>>>>> release.
>>>>> 
>>>>> There
>>>>> 
>>>>> are, however, about 90+ JIRAs assigned to 1.1.0 which
>>>>> 
>>>>> are
>>>>> 
>>>>> open.
>>>>> 
>>>>> I'm
>>>>> 
>>>>> going to go through them and remove fix versions where
>>>>> 
>>>>> appropriate.
>>>>> 
>>>>> 
>>>>> I'm happy to take on RM task for this release though if
>>>>> 
>>>>> someone
>>>>> 
>>>>> else
>>>>> 
>>>>> would like to take that on please advise.
>>>>> 
>>>>> Thanks
>>>>> Joe
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Cheers,
>>>>> 
>>>>> Edgardo
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Cheers,
>>>>> 
>>>>> Edgardo
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Cheers,
>>>>> 
>>>>> Edgardo
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Cheers,
>>>>> 
>>>>> Edgardo
>>>>> 
>>>>> Sent from Gmail Mobile
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 


Mime
View raw message