nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff <jtsw...@gmail.com>
Subject Re: [DISCUSS] Closing in on a release of NiFi 1.8.0?
Date Tue, 16 Oct 2018 20:52:12 GMT
A quick update on RC1 for NiFi 1.8.0.  There are some L&N issues that are
currently being worked, so I'd like to hold off on creating the RC until
they are resolved.  Sorry for the delay!

On Tue, Oct 16, 2018 at 3:46 PM Nathan Gough <thenatog@gmail.com> wrote:

> So, taking a closer look at this PR and doing some further testing I've
> found that upgrading Guava is going to require some more careful
> assessment, as the changes will affect core NiFi functionality around
> clustering. I will hold off on this change until after the release to have
> more time to upgrade and test.
>
> Nathan
>
> On 10/16/18, 10:58 AM, "Nathan Gough" <thenatog@gmail.com> wrote:
>
>     Hi Mike,
>
>     Sure I can look at fixing up PR-2977 today.
>
>     Nathan
>
>     On 10/16/18, 6:13 AM, "Mike Thomsen" <mikerthomsen@gmail.com> wrote:
>
>         Does 5562 need to be addressed in 1.8?
>
>         https://github.com/apache/nifi/pull/2977
>
>         On Mon, Oct 15, 2018 at 6:33 PM Jeff <jtswork@gmail.com> wrote:
>
>         > NiFi Devs,
>         >
>         > The Release page [1] for 1.8.0 now reports that all issues are
> done!  I'd
>         > like to start the release candidate preparation tomorrow, around
> 1200 EST.
>         >
>         > Thanks to everyone for all the great work that's been done!  196
> issues
>         > resolved in this version with some great new features!
>         >
>         > [1]
> https://issues.apache.org/jira/projects/NIFI/versions/12343482
>         >
>         > On Mon, Oct 15, 2018 at 7:30 AM Sivaprasanna <
> sivaprasanna246@gmail.com>
>         > wrote:
>         >
>         > > Great. Thanks. :)
>         > >
>         > > -
>         > > Sivaprasanna
>         > >
>         > > On Mon, Oct 15, 2018 at 7:09 AM Koji Kawamura <
> ijokarumawak@gmail.com>
>         > > wrote:
>         > >
>         > > > Jeff, Sivasprasanna,
>         > > >
>         > > > NIFI-5698 (PR3073) Fixing DeleteAzureBlob bug is merged.
>         > > >
>         > > > Thanks,
>         > > > Koji
>         > > > On Mon, Oct 15, 2018 at 10:18 AM Koji Kawamura <
> ijokarumawak@gmail.com
>         > >
>         > > > wrote:
>         > > > >
>         > > > > Thank you for the fix Sivaprasanna,
>         > > > > I have Azure account. Reviewing it now.
>         > > > >
>         > > > > Koji
>         > > > > On Sun, Oct 14, 2018 at 11:21 PM Jeff <jtswork@gmail.com>
> wrote:
>         > > > > >
>         > > > > > Sivaprasanna,
>         > > > > >
>         > > > > > Thanks for submitting a pull request for that issue!
> Later today
>         > or
>         > > > > > tomorrow I'll have to check to see if I've already used
> up my
>         > > free-tier
>         > > > > > access to Azure.  If I still have access, I can review
> your PR and
>         > > > we'll
>         > > > > > get it into 1.8.0.
>         > > > > >
>         > > > > > On Sun, Oct 14, 2018 at 4:30 AM Sivaprasanna <
>         > > > sivaprasanna246@gmail.com>
>         > > > > > wrote:
>         > > > > >
>         > > > > > > All - Just found one bug with DeleteAzureBlobStorage
> processor.
>         > It
>         > > > was
>         > > > > > > shared by one user on StackOverflow [1] and I later
> confirmed it.
>         > > It
>         > > > looks
>         > > > > > > to be introduced by NIFI-4199. I have created a
Jira
> [2] and made
>         > > the
>         > > > > > > necessary changes (not huge, just few lines) and
> raised a PR
>         > [3]. I
>         > > > think,
>         > > > > > > if we can spend a little time in getting it reviewed,
> we can mark
>         > > it
>         > > > for
>         > > > > > > 1.8.0. Thoughts?
>         > > > > > >
>         > > > > > > [1] -
>         > > > > > >
>         > > > > > >
>         > > >
>         > >
>         >
> https://stackoverflow.com/questions/52766991/apache-nifi-deleteazureblobstorage-processor-is-throwing-an-error
>         > > > > > > [2] - https://issues.apache.org/jira/browse/NIFI-5698
>         > > > > > > [3] - https://github.com/apache/nifi/pull/3073
>         > > > > > >
>         > > > > > > -
>         > > > > > > Sivaprasanna
>         > > > > > >
>         > > > > > > On Fri, Oct 12, 2018 at 9:05 PM Mike Thomsen <
>         > > mikerthomsen@gmail.com
>         > > > >
>         > > > > > > wrote:
>         > > > > > >
>         > > > > > > > 4811 should be ready for review now. Rebased
and
> cleaned it up
>         > > > with a
>         > > > > > > full
>         > > > > > > > listing of the Spring dependencies.
>         > > > > > > >
>         > > > > > > > On Fri, Oct 12, 2018 at 11:23 AM Joe Witt
<
> joe.witt@gmail.com>
>         > > > wrote:
>         > > > > > > >
>         > > > > > > > > Jeff,
>         > > > > > > > >
>         > > > > > > > > I think for anything not tagged to 1.8.0
we just
> keep
>         > rolling.
>         > > > For
>         > > > > > > > > anything tagged 1.8.0 that should not
be we should
> remove it
>         > > > until
>         > > > > > > > > ready.  For things tagged to 1.8.0 that
cannot be
> moved we
>         > > should
>         > > > > > > > > resolve.  For the tagged 1.8.0 section
you had.
>         > > > > > > > >
>         > > > > > > > >    - NIFI-4811 <
>         > > https://issues.apache.org/jira/browse/NIFI-4811>
>         > > > -
>         > > > > > > Use a
>         > > > > > > > >    newer version of spring-data-redis
>         > > > > > > > >    - PR 2856 <
> https://github.com/apache/nifi/pull/2856>
>         > > > > > > > > *This needs to be resolved by either
reverting the
> commit or
>         > > > ensuring
>         > > > > > > > > L&N accurately reflects all.  We
have to do this
> always and
>         > for
>         > > > every
>         > > > > > > > > nar.  The process isnt easy or fun but
it is
> necessary to
>         > > produce
>         > > > > > > > > valid ASF releases.  Landing commits
which change
>         > dependencies
>         > > > > > > > > requires this due diligence.  Now, we've
put a lot
> of energy
>         > > into
>         > > > > > > > > updating Spring dependencies because
some older
> Spring libs
>         > had
>         > > > > > > > > vulnerabilities which while we likely
aren't
> exposed to them
>         > we
>         > > > want
>         > > > > > > > > to fix in due course.  So reverting may
require
> more analysis
>         > > > than if
>         > > > > > > > > we were just get L&N fixed with this
new change.
> I commented
>         > > on
>         > > > the
>         > > > > > > > > JIRA.  But this needs to be resolved.
>         > > > > > > > >
>         > > > > > > > >
>         > > > > > > > >    - NIFI-5426 <
>         > > https://issues.apache.org/jira/browse/NIFI-5426>
>         > > > - Use
>         > > > > > > > >    NIO.2 API for ListFile to avoid multiple
disk
> reads
>         > > > > > > > >       - PR 2889 <
> https://github.com/apache/nifi/pull/2889>
>         > > > > > > > > *This just needed to be marked resolved.
 The
> commit went in
>         > > the
>         > > > day
>         > > > > > > > > after we cut 1.7.1.  So this one is sorted.
>         > > > > > > > >
>         > > > > > > > >    - NIFI-5448 <
>         > > https://issues.apache.org/jira/browse/NIFI-5448>
>         > > > -
>         > > > > > > > Failed
>         > > > > > > > >    EL date parsing live-locks processors
without a
> failure
>         > > > relationship
>         > > > > > > > > * The commit needs to be reverted.  I'm
working on
> that now.
>         > > > Once the
>         > > > > > > > > discsusion/concerns are addressed this
can get
> dealt with.
>         > > > > > > > >
>         > > > > > > > >    - NIFI-5665 <
>         > > https://issues.apache.org/jira/browse/NIFI-5665>
>         > > > -
>         > > > > > > > Upgrade
>         > > > > > > > >    io.netty dependencies
>         > > > > > > > > * This looks important to get resolved
if possible
> as old
>         > netty
>         > > > libs
>         > > > > > > > > are on the list of things with vulnerabilities.
>         > > > > > > > >
>         > > > > > > > >    - NIFI-5686 <
>         > > https://issues.apache.org/jira/browse/NIFI-5686>
>         > > > -
>         > > > > > > Test
>         > > > > > > > >    failure in TestStandardProcessScheduler
>         > > > > > > > >    - PR 3062 <
> https://github.com/apache/nifi/pull/3062>
>         > > > > > > > > * This has a PR but a test, possibly
two, failed
> in one of
>         > the
>         > > > travis
>         > > > > > > > > runs and it is clearly related.  I ignored
one of
> those tests
>         > > in
>         > > > a
>         > > > > > > > > previous run.  We must deal with brittle
tests.
> But the
>         > > > underlying
>         > > > > > > > > problem is important to solve here so
either the
> tests needs
>         > > > improved
>         > > > > > > > > or we still have an issue.  Not clear
but worth
> some focus.
>         > > > > > > > >
>         > > > > > > > > note: I intend to reference updates to
libraries
> that have
>         > > known
>         > > > > > > > > vulnerabilities and do so in a far less
subtle
> manner than we
>         > > > had.  We
>         > > > > > > > > aren't acknowledging that NiFi is or
exposes
> vulnerabilities
>         > > but
>         > > > we
>         > > > > > > > > are and should be clear when we're updating
> dependencies that
>         > > do
>         > > > have
>         > > > > > > > > them (even if we're not exposed to them)
so that
> some of
>         > these
>         > > > commits
>         > > > > > > > > aren't so mysterious.  It creates far
more
> confusion than is
>         > > > worth.
>         > > > > > > > > We still will follow the ASF/NiFi security
> handling policy
>         > but
>         > > I
>         > > > no
>         > > > > > > > > longer intend to treat due course dependency
> updates as if
>         > they
>         > > > need
>         > > > > > > > > to be a secret.
>         > > > > > > > >
>         > > > > > > > > Thanks
>         > > > > > > > > Joe
>         > > > > > > > >
>         > > > > > > > >
>         > > > > > > > > On Fri, Oct 12, 2018 at 3:32 AM Jeff
<
> jtswork@gmail.com>
>         > > wrote:
>         > > > > > > > > >
>         > > > > > > > > > Hello everyone!  Next week is probably
a good
> timeframe to
>         > > aim
>         > > > for a
>         > > > > > > > > > release candidate, with two major
feature PRs
> recently
>         > merged
>         > > > to
>         > > > > > > > master:
>         > > > > > > > > >
>         > > > > > > > > >    - NIFI-5516 <
>         > > > https://issues.apache.org/jira/browse/NIFI-5516> -
>         > > > > > > > Allow
>         > > > > > > > > >    data in a Connection to be Load-Balanced
> across cluster
>         > > > > > > > > >    - NIFI-5585 <
>         > > > https://issues.apache.org/jira/browse/NIFI-5585> -
>         > > > > > > > > Prepare
>         > > > > > > > > >    Nodes to be Offloaded
>         > > > > > > > > >
>         > > > > > > > > >
>         > > > > > > > > > To recap, here's a list of other
JIRAs mentioned
> in this
>         > > > thread:
>         > > > > > > > > >
>         > > > > > > > > >    - NIFI-5402 <
>         > > > https://issues.apache.org/jira/browse/NIFI-5402> -
>         > > > > > > > > Reduce
>         > > > > > > > > >    artifact size by only building
.zip archive
>         > > > > > > > > >    - NIFI-5462 <
>         > > > https://issues.apache.org/jira/browse/NIFI-5462> -
>         > > > > > > > > Refactor
>         > > > > > > > > >    TLS Toolkit
>         > > > > > > > > >    - NIFI-5485 <
>         > > > https://issues.apache.org/jira/browse/NIFI-5485> -
>         > > > > > > > > Enable
>         > > > > > > > > >    TLS Toolkit (client/server) to
sign
> certificates with
>         > > > external CA
>         > > > > > > > > >    certificate
>         > > > > > > > > >    - NIFI-5537 <
>         > > > https://issues.apache.org/jira/browse/NIFI-5537> -
>         > > > > > > > > Create
>         > > > > > > > > >    Neo4J cypher execution processor
>         > > > > > > > > >    - PR 2956 <
> https://github.com/apache/nifi/pull/2956>
>         > > > > > > > > >       - Mike Thomsen, this was the
specific JIRA
> to which
>         > you
>         > > > were
>         > > > > > > > > >       referring, right?
>         > > > > > > > > >    - NIFI-5582 <
>         > > > https://issues.apache.org/jira/browse/NIFI-5582> -
>         > > > > > > > > Integrate
>         > > > > > > > > >    legacy behavior of HashAttribute
into
>         > > > CryptographicHashAttribute
>         > > > > > > > > >
>         > > > > > > > > >
>         > > > > > > > > > These JIRAs are marked with a fix
version of
> 1.8.0 that are
>         > > not
>         > > > > > > > currently
>         > > > > > > > > > resolved:
>         > > > > > > > > >
>         > > > > > > > > >    - NIFI-4811 <
>         > > > https://issues.apache.org/jira/browse/NIFI-4811> -
>         > > > > > > > Use a
>         > > > > > > > > >    newer version of spring-data-redis
>         > > > > > > > > >    - PR 2856 <
> https://github.com/apache/nifi/pull/2856>
>         > > > > > > > > >    - NIFI-5426 <
>         > > > https://issues.apache.org/jira/browse/NIFI-5426> -
>         > > > > > > Use
>         > > > > > > > > >    NIO.2 API for ListFile to avoid
multiple disk
> reads
>         > > > > > > > > >       - PR 2889 <
> https://github.com/apache/nifi/pull/2889>
>         > > > > > > > > >    - NIFI-5448 <
>         > > > https://issues.apache.org/jira/browse/NIFI-5448> -
>         > > > > > > > > Failed
>         > > > > > > > > >    EL date parsing live-locks processors
without
> a failure
>         > > > > > > relationship
>         > > > > > > > > >    - NIFI-5665 <
>         > > > https://issues.apache.org/jira/browse/NIFI-5665> -
>         > > > > > > > > Upgrade
>         > > > > > > > > >    io.netty dependencies
>         > > > > > > > > >    - NIFI-5686 <
>         > > > https://issues.apache.org/jira/browse/NIFI-5686> -
>         > > > > > > > Test
>         > > > > > > > > >    failure in TestStandardProcessScheduler
>         > > > > > > > > >    - PR 3062 <
> https://github.com/apache/nifi/pull/3062>
>         > > > > > > > > >
>         > > > > > > > > >
>         > > > > > > > > > On Thu, Oct 4, 2018 at 6:39 AM Mike
Thomsen <
>         > > > mikerthomsen@gmail.com>
>         > > > > > > > > wrote:
>         > > > > > > > > >
>         > > > > > > > > > > That's a fair point. Only thing
I could add
> there is
>         > that I
>         > > > think
>         > > > > > > we
>         > > > > > > > > should
>         > > > > > > > > > > consider a targeted burn down
on the PR list
> as part of
>         > > 1.9.
>         > > > There
>         > > > > > > > are
>         > > > > > > > > a
>         > > > > > > > > > > lot of PRs from the last several
months that
> would be
>         > good
>         > > > > > > candidates
>         > > > > > > > > to
>         > > > > > > > > > > see if we can close them out
like MarkLogic
> and Pulsar.
>         > > > > > > > > > >
>         > > > > > > > > > > On Wed, Oct 3, 2018 at 10:34
PM Joe Witt <
>         > > joe.witt@gmail.com
>         > > > >
>         > > > > > > wrote:
>         > > > > > > > > > >
>         > > > > > > > > > > > Mike,
>         > > > > > > > > > > >
>         > > > > > > > > > > > Processors in particularly
are among the
> toughest at
>         > this
>         > > > point.
>         > > > > > > > We
>         > > > > > > > > > > > have very very little
headroom on dependency
> size for
>         > the
>         > > > full
>         > > > > > > > build
>         > > > > > > > > > > > size that we upload to
ASF infra and
> mirrors.  That and
>         > > the
>         > > > > > > license
>         > > > > > > > > > > > review work involved in
each...
>         > > > > > > > > > > >
>         > > > > > > > > > > > We should really create
a way to publish
> processors on
>         > > more
>         > > > > > > > frequent,
>         > > > > > > > > > > > irregular intervals where
the release work
> and
>         > size/etc..
>         > > > are far
>         > > > > > > > > less
>         > > > > > > > > > > > problematic.  We have
another discuss thread
> on that so
>         > > > I'll
>         > > > > > > leave
>         > > > > > > > it
>         > > > > > > > > > > > there for discussion.
 I do share your view
> that this
>         > > > processor
>         > > > > > > > > (among
>         > > > > > > > > > > > several others outstanding)
would be really
> useful but
>         > i
>         > > am
>         > > > > > > > > definitely
>         > > > > > > > > > > > thinking we should keep
release pace up.
> Release more
>         > > > > > > > > often...release
>         > > > > > > > > > > > processors separately,
etc..
>         > > > > > > > > > > >
>         > > > > > > > > > > > Thanks
>         > > > > > > > > > > > Joe
>         > > > > > > > > > > > On Wed, Oct 3, 2018 at
9:30 PM Mike Thomsen <
>         > > > > > > > mikerthomsen@gmail.com>
>         > > > > > > > > > > > wrote:
>         > > > > > > > > > > > >
>         > > > > > > > > > > > > I would like to see
the Neo4J work that
> mans2singh is
>         > > > doing get
>         > > > > > > > > > > included.
>         > > > > > > > > > > > > Being able to at
least partially support a
> popular
>         > > graph
>         > > > > > > database
>         > > > > > > > > would
>         > > > > > > > > > > > be
>         > > > > > > > > > > > > a nice feather in
our cap.
>         > > > > > > > > > > > >
>         > > > > > > > > > > > > On Wed, Oct 3, 2018
at 5:12 PM Andy
> LoPresto <
>         > > > > > > > alopresto@apache.org
>         > > > > > > > > >
>         > > > > > > > > > > > wrote:
>         > > > > > > > > > > > >
>         > > > > > > > > > > > > > I am currently
working on a TLS Toolkit
> refactor
>         > > > (NIFI-5462 &
>         > > > > > > > > > > > NIFI-5485)
>         > > > > > > > > > > > > > and HashAttribute
updates (NIFI-5582). I
> believe
>         > > there
>         > > > are a
>         > > > > > > > > couple
>         > > > > > > > > > > > upgrade
>         > > > > > > > > > > > > > PRs open, and
I would really like to see
> NIFI-5402
>         > > (no
>         > > > > > > .tar.gz
>         > > > > > > > > in the
>         > > > > > > > > > > > > > build) tackled
for this release as well.
>         > > > > > > > > > > > > >
>         > > > > > > > > > > > > >
>         > > > > > > > > > > > > > Andy LoPresto
>         > > > > > > > > > > > > > alopresto@apache.org
>         > > > > > > > > > > > > > *alopresto.apache@gmail.com
<
>         > > > alopresto.apache@gmail.com>*
>         > > > > > > > > > > > > > PGP Fingerprint:
70EC B3E5 98A6 5A3F
> D3C4  BACE
>         > 3C6E
>         > > > F65B
>         > > > > > > 2F7D
>         > > > > > > > > EF69
>         > > > > > > > > > > > > >
>         > > > > > > > > > > > > > On Oct 3, 2018,
at 11:16 AM, Joe Witt <
>         > > > joewitt@apache.org>
>         > > > > > > > > wrote:
>         > > > > > > > > > > > > >
>         > > > > > > > > > > > > > Jeff - thanks
again for volunteering.  I
> just went
>         > > > through
>         > > > > > > the
>         > > > > > > > > open
>         > > > > > > > > > > > > > items tagged
to 1.8.0 to try and shake
> some loose,
>         > > > close down
>         > > > > > > > > ones
>         > > > > > > > > > > > > > that appear
to be done but forgotten,
> and initiate
>         > > > resolution
>         > > > > > > > on
>         > > > > > > > > one
>         > > > > > > > > > > > > > that is in a
dangling state.
>         > > > > > > > > > > > > >
>         > > > > > > > > > > > > > Another very
nice release shaping up
> here.  All the
>         > > > work
>         > > > > > > around
>         > > > > > > > > load
>         > > > > > > > > > > > > > balancing and
node offloading is awesome.
>         > > > > > > > > > > > > >
>         > > > > > > > > > > > > > Thanks
>         > > > > > > > > > > > > > On Wed, Oct
3, 2018 at 2:06 PM Jeff <
>         > > > jstorck@apache.org>
>         > > > > > > > wrote:
>         > > > > > > > > > > > > >
>         > > > > > > > > > > > > >
>         > > > > > > > > > > > > > It looks like
we're getting close to a
> point where
>         > we
>         > > > could
>         > > > > > > > > release
>         > > > > > > > > > > > NiFi
>         > > > > > > > > > > > > > 1.8.0The release
tracking page for
> version 1.8.0
>         > [1]
>         > > > shows 3
>         > > > > > > > "in
>         > > > > > > > > > > > progress"
>         > > > > > > > > > > > > > and 9 "to do"
issues. In addition to
> what has been
>         > > > tagged
>         > > > > > > with
>         > > > > > > > a
>         > > > > > > > > fix
>         > > > > > > > > > > > > > version of 1.8.0,
it looks like
> NIFI-5516 and
>         > > > NIFI-5585 are
>         > > > > > > > > close to
>         > > > > > > > > > > > > > completion.
>         > > > > > > > > > > > > >
>         > > > > > > > > > > > > > Are there other
JIRAs that the community
> considers
>         > > > necessary
>         > > > > > > > for
>         > > > > > > > > the
>         > > > > > > > > > > > > > release that
are close to being
> resolved, with the
>         > > > goal of
>         > > > > > > > > getting a
>         > > > > > > > > > > > > > release candidate
out in the next couple
> of weeks?
>         > > > > > > > > > > > > >
>         > > > > > > > > > > > > > I'm happy to
perform the release manager
> duties!
>         > > > > > > > > > > > > >
>         > > > > > > > > > > > > > [1]
>         > > > > > > > >
>         > https://issues.apache.org/jira/projects/NIFI/versions/12343482
>         > > > > > > > > > > > > >
>         > > > > > > > > > > > > >
>         > > > > > > > > > > > > >
>         > > > > > > > > > > >
>         > > > > > > > > > >
>         > > > > > > > >
>         > > > > > > >
>         > > > > > >
>         > > >
>         > >
>         >
>
>
>
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message