nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <joe.w...@gmail.com>
Subject Re: MiNiFi commits for h2o
Date Mon, 04 May 2020 16:13:16 GMT
Yeah for the three/commit James replied on in that link - i dont see how we
can keep those.  They import from something that I doubt very much is
compatible with ALv2/ASF guidelines.

On Mon, May 4, 2020 at 12:01 PM Marc Parisi <marc@parisi.io> wrote:

> Afternoon Joe,
>
> "So in the case of minificpp we are certain there is no L&N concern
> regarding either our source OR convenience binary?"
>
> I believe that to be the case; however, per
> https://github.com/apache/nifi/pull/4242#issuecomment-622655276 I think
> that maybe this is moot and the author can convert those processors in
> MiNiFI to use that open source python lib. I'm not sure if he's on this
> chain so I will follow up on the MiNiFi tickets to ensure that is the case
> when I am able to log in again ( have to fix some account issue ).
>
> "Lastly, why are the JIRAs still open if this is merged?  Can you please
> close them and assign to the proper fix version if these are 'done'?"
>
> Sorry, when I am able to log in ( account issue ) and make the above
> comment I will close the tx. I would prefer the fix be a follow on so this
> is all tracked. The ticket fell off the radar ( as did the account issue,
> to be frank ), so when that is resolved I will take these actions. You know
> me Joe, I'm like a dog who sees a squirrel, but I've put a note on my
> laptop to serve as a reminder.
>
> Thanks,
> Marc
>
> On Mon, May 4, 2020 at 11:49 AM Joe Witt <joe.witt@gmail.com> wrote:
>
> > Marc
> >
> > I'm not opposed to it staying if there is zero license issue.  I only
> > mentioned the maintenance as a 'additional thing to consider' - that is
> > more of a discussion item.  L&N isn't discussion - it is hard rule.
> >
> > Ok so in the minificpp case you're saying "there is no source or binary
> > dependency (that we'd package)" that isn't entirely ALv2/ASF compatible
> and
> > so if people want to use this they can do so by adding in the 'h2o' bits
> at
> > their own effort, right?  In this line of thinking I still don't love it
> > but it is arguably not a lot different that our components which talk to
> > services you have to have a paid or otherwise agreement with like Cloud
> > services offering by AWS, Azure, GCP, etc..
> >
> > So in the case of minificpp we are certain there is no L&N concern
> > regarding either our source OR convenience binary?
> >
> > Lastly, why are the JIRAs still open if this is merged?  Can you please
> > close them and assign to the proper fix version if these are 'done'?
> >
> > Thanks
> >
> > On Mon, May 4, 2020 at 11:46 AM Marc Parisi <phrocker@apache.org> wrote:
> >
> > > "We should find a way for vendors to offer extension points like this
> > > without having to take on the burden of maintenance. How can we
> possibly
> > do
> > > this well?  We're learning this lesson the hard way in NiFi itself and
> > this
> > > is why the registry is being formulated."
> > >
> > > I think there is a fine line when walking vendor tie-in. Having the
> > > registry allows for an arm's length agreement with the Apache
> community.
> > I
> > > know others will disagree but my opinion is that extension points that
> > > don't introduce licensing concerns from libraries should be fair game
> but
> > > the fact that a paid product requires a license for that code to be
> > useful
> > > should not be a non-starter -- if that makes sense. I believe I stated,
> > "I
> > > don't have a strong argument" as I don't really have a strong opinion (
> > and
> > > certainly can be wrong too ) -- so I'm interested in hearing others in
> > > regards to this.
> > >
> > >
> > >
> > > On Mon, May 4, 2020 at 11:38 AM Marc Parisi <phrocker@apache.org>
> wrote:
> > >
> > > > Hi Joe,
> > > >
> > > > When merging I did not use the NiFi processors to test as I already
> > have
> > > > tooling around H20 driven ML in my home. While I don't admittedly use
> > it
> > > > very often, I thought the python processors were quite cool and could
> > > > certainly be useful for others. My rationale is that the dependencies
> > are
> > > > not built into source or binary for MiNiFi C++. I would agree on the
> > Java
> > > > side that this would be unavoidable. While it would be preferential
> to
> > > have
> > > > the NiFi analogue I do not think it would be required when
> considering
> > > the
> > > > MiNiFi C++ processors; however, James should be the arbiter of that
> > > > decision.
> > > >
> > > > While I disagree that the MiNiFi portion should be removed, I don't
> > have
> > > a
> > > > strong argument to keep or remove the MiNiFI C++ code so I'd be happy
> > to
> > > > revert if the community feels the need to.
> > > >
> > > > Thanks,
> > > > Marc
> > > >
> > > > On Mon, May 4, 2020 at 10:50 AM Joe Witt <joe.witt@gmail.com> wrote:
> > > >
> > > >> Team
> > > >>
> > > >> The below two commits raise serious concerns.
> > > >>
> > > >> I want to be clear here and point out that h2o is cool.  Having such
> > > >> integration is a neat concept and idea and one that certainly
> warrants
> > > >> determination on how best to do so.
> > > >>
> > > >> My issue is with these two commits as it relates to licensing and
> > > >> maintenance.
> > > >>
> > > >> License:
> > > >> We should support vendors like this wanting to bring their
> > capabilities
> > > >> into NiFi generally sure. But the licensing and mode of use is
> > critical.
> > > >> In talking about this with the contributor for NiFi as well it is
> > clear
> > > >> that at least some or important portions of this require the user
to
> > > have
> > > >> a
> > > >> 'driverless ai license' so they can include their jar or build
> > pipelines
> > > >> or
> > > >> whatnot.  Thus it isn't usable without that first. So it might be
> the
> > > case
> > > >> that this stuff is source dependent only on ASF compatible licenses
> in
> > > >> terms of source - but it certainly doesn't seem to be true in terms
> of
> > > >> binary dependencies. Where in the PR or JIRA is there any discussion
> > or
> > > >> review of the licensing?  I see plenty from James to believe this
> > needs
> > > to
> > > >> be reverted.  Any contrary info?
> > > >> https://github.com/apache/nifi/pull/4242#issuecomment-622654527
> > > >>
> > > >>
> > > >> Maintenance:
> > > >> We should find a way for vendors to offer extension points like this
> > > >> without having to take on the burden of maintenance. How can we
> > possibly
> > > >> do
> > > >> this well?  We're learning this lesson the hard way in NiFi itself
> and
> > > >> this
> > > >> is why the registry is being formulated.
> > > >>
> > > >> JIRA/hygiene:
> > > >> https://issues.apache.org/jira/browse/MINIFICPP-1199
> > > >> and
> > > >> https://issues.apache.org/jira/browse/MINIFICPP-1201
> > > >> Both are open.  Yet we've merged commits that claim to be against
> > each.
> > > >>
> > > >> I believe both of these commits need to be reverted as I do not
> > believe
> > > >> the
> > > >> licensing considerations have been addressed.  I'd like to see
> > > discussion
> > > >> on the above maintenance concern as well but that is less pressing.
> > > >>
> > > >>
> > > >> Thanks
> > > >> Joe
> > > >>
> > > >>
> > > >> commit 7206c62240647520cf35649868d5d87903a256c2
> > > >> Author: James Medel <jamesmedel94@gmail.com>
> > > >> Date:   Wed Apr 29 12:38:04 2020 -0700
> > > >>
> > > >>     MINIFI-1201: Integrate H2O Driverless AI MSP in MiNFi (#766)
> > > >>
> > > >>     MINIFI-1201: Integrate H2O Driverless AI MSP in MiNFi (#766)
> > > >>
> > > >> commit 6e5f96518764df7791519c0ee625a94a207ddc69
> > > >> Author: James Medel <jamesmedel94@gmail.com>
> > > >> Date:   Wed Apr 29 12:37:00 2020 -0700
> > > >>
> > > >>     MINIFI-1199: Integrate H2O Driverless AI PSP in MiNiFi (#763)
> > > >>
> > > >>     * MINIFI-1199: Integrate H2O Driverless AI in MiNiFi
> > > >>
> > > >>     MiNiFi C++ and H2O Driverless AI Integration via Custom Python
> > > >> Processors:
> > > >>     Integrates MiNiFi C++ with H2O's Driverless AI by Using
> Driverless
> > > >> AI's
> > > >> Python Scoring Pipeline and MiNiFi's Custom Python Processors. Uses
> > the
> > > >> Python Processors to execute the Python Scoring Pipeline scorer to
> do
> > > >> batch
> > > >> scoring and real-time scoring for one or more predicted labels on
> test
> > > >> data
> > > >> in the incoming flow file content. I would like to contribute my
> > > >> processors
> > > >> to MiNiFi C++ as a new feature.
> > > >>
> > > >>     * Update H2oPspScoreBatches processor
> > > >>
> > > >>     This update includes passing "index=False" to
> > > >> "batch_scores_df.to_string(index=False)". By updating this line of
> > code,
> > > >> we
> > > >> tell pandas to not include the DataFrame index when converting the
> > > >> DataFrame to a string. The reason for this update is that we don't
> > want
> > > >> this extra column pandas tries to add to our predictions frame, we
> > only
> > > >> want the predictions. Thus, later when the predictions get saved to
> a
> > > csv
> > > >> file, it will only include the predictions.
> > > >>
> > > >>     * Moved ConvertDsToCsv to h2o base dir
> > > >>
> > > >>     Since this ConvertDsToCsv python processor is used by the
> > > >>     Python Scoring Pipeline and MOJO Scoring Pipeline processors,
> > > >>     I moved ConvertDsToCsv to h2o base dir for easier access to it.
> > > >>
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message