nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Rosander <bryanrosan...@gmail.com>
Subject Re: [VOTE] Release Apache NiFi MiNiFi 0.0.1 (RC1)
Date Sat, 09 Jul 2016 14:53:10 GMT
Hey Pierre,

I agree with you on both points.

1. We should decide whether CRON support is worth the extra footprint and
overhead of quartz.  If so we should add it, if not we should catch that
during validation.

2. I've already written up an issue for it.

https://issues.apache.org/jira/browse/MINIFI-46

Thanks,
Bryan
On Jul 9, 2016 10:07 AM, "Pierre Villard" <pierre.villard.fr@gmail.com>
wrote:

> Hi guys,
>
> +0 (non-binding)
>
> Checked signature and checksum. Built sources without issue. Used toolkit
> to convert template. And ran a template to exchange data between a MiNiFi
> instance and a NiFi instance. Great !!
> (under Windows 10 environment)
>
> Few remarks though (sorry if it has already been mentioned):
>
> 1. CRON driven support
>
> It seems to me that CRON driven processors are not supported (following
> stack trace when running MiNiFi with a template with a CRON driven
> processor):
>
> 2016-07-09 15:18:08,492 ERROR [main] org.apache.nifi.minifi.MiNiFi Failure
> to launch MiNiFi due to java.lang.NoClassDefFoundError:
> org/quartz/CronExpression
> java.lang.NoClassDefFoundError: org/quartz/CronExpression
>     at
>
> org.apache.nifi.controller.StandardProcessorNode.setScheduldingPeriod(StandardProcessorNode.java:446)
> ~[nifi-framework-core-0.6.1.jar:0.6.1]
>     at
>
> org.apache.nifi.controller.StandardFlowSynchronizer.updateProcessor(StandardFlowSynchronizer.java:721)
> ~[nifi-framework-core-0.6.1.jar:0.6.1]
>     at
>
> org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:776)
> ~[nifi-framework-core-0.6.1.jar:0.6.1]
>     at
>
> org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:271)
> ~[nifi-framework-core-0.6.1.jar:0.6.1]
>     at
>
> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1287)
> ~[nifi-framework-core-0.6.1.jar:0.6.1]
>     at
>
> org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:72)
> ~[nifi-framework-core-0.6.1.jar:0.6.1]
>     at
>
> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:629)
> ~[nifi-framework-core-0.6.1.jar:0.6.1]
>     at
>
> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:479)
> ~[nifi-framework-core-0.6.1.jar:0.6.1]
>     at org.apache.nifi.minifi.MiNiFiServer.start(MiNiFiServer.java:75)
> ~[minifi-framework-core-0.0.1.jar:0.0.1]
>     at org.apache.nifi.minifi.MiNiFi.<init>(MiNiFi.java:134)
> ~[minifi-runtime-0.0.1.jar:0.0.1]
>     at org.apache.nifi.minifi.MiNiFi.main(MiNiFi.java:233)
> ~[minifi-runtime-0.0.1.jar:0.0.1]
> Caused by: java.lang.ClassNotFoundException: org.quartz.CronExpression
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> ~[na:1.8.0_74]
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_74]
>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> ~[na:1.8.0_74]
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_74]
>     ... 11 common frames omitted
>
> If it is expected, it'd be nice to have a warning while converting
> templates.
>
>
> 2. Relationships conversion
>
> I have a processor connected (for both success and failure relationships)
> to a remote process group in my original template (XML). When converting
> into a YAML template, only the failure relationship is here and I get the
> following error:
>
> 2016-07-09 15:24:58,921 ERROR [main] o.a.nifi.groups.StandardProcessGroup
> Unable to start AttributesToJSON[id=AttributesToJSON] due to {}
> java.lang.IllegalStateException: Processor AttributesToJSON is not in a
> valid state due to ['Relationship success' is invalid because Relationship
> 'success' is not connected to any component and is not auto-terminated]
>     at
>
> org.apache.nifi.controller.StandardProcessorNode.start(StandardProcessorNode.java:1235)
> ~[nifi-framework-core-0.6.1.jar:0.6.1]
>     at
>
> org.apache.nifi.controller.scheduling.StandardProcessScheduler.startProcessor(StandardProcessScheduler.java:321)
> ~[nifi-framework-core-0.6.1.jar:0.6.1]
>     at
>
> org.apache.nifi.groups.StandardProcessGroup.startProcessor(StandardProcessGroup.java:1047)
> ~[nifi-framework-core-0.6.1.jar:0.6.1]
>     at
>
> org.apache.nifi.groups.StandardProcessGroup.startProcessing(StandardProcessGroup.java:268)
> ~[nifi-framework-core-0.6.1.jar:0.6.1]
>     at org.apache.nifi.minifi.MiNiFiServer.start(MiNiFiServer.java:77)
> [minifi-framework-core-0.0.1.jar:0.0.1]
>     at org.apache.nifi.minifi.MiNiFi.<init>(MiNiFi.java:134)
> [minifi-runtime-0.0.1.jar:0.0.1]
>     at org.apache.nifi.minifi.MiNiFi.main(MiNiFi.java:233)
> [minifi-runtime-0.0.1.jar:0.0.1]
>
> I had to manually modify the YAML configuration to get things working.
> Maybe I missed something here.
>
> In any case, not a big deal from my point of view. Let me know if I should
> raised a JIRA.
>
> For interested people, I wrote a short article about what I did to test
> MiNiFi (first time I was looking into it):
> https://pierrevillard.com/2016/07/09/apache-nifi-minifi-is-almost-out/
>
> Great job!!
> Pierre
>
>
> 2016-07-08 23:26 GMT+02:00 Mark Payne <markap14@hotmail.com>:
>
> > And since i originally replied to the helper thread instead of the VOTE
> > thread...
> >
> > +1 (binding)
> >
> > Verified signature and checksums. Build completed without issue. Was able
> > to use the minifi-toolkit to convert
> > a nifi template and run it without any problem. All worked perfectly!
> > Great job, guys!
> >
> >
> > > On Jul 8, 2016, at 1:46 PM, Aldrin Piri <aldrinpiri@gmail.com> wrote:
> > >
> > > +1, binding
> > >
> > > Build, contrib, and associated documentation looks good.
> > >
> > > Hashes and signatures check out.
> > >
> > > Was able to use the generated assemblies to perform some tests and
> > > communicate with a NiFi instance.
> > >
> > > ---
> > >
> > > Folks,
> > >
> > > Thanks for all the feedback.  I've created tickets to capture all of
> the
> > > points made.  Much appreciated and happy to have some additional eyes
> on
> > > the effort.
> > >
> > >
> > > On Fri, Jul 8, 2016 at 1:27 PM, Andy LoPresto <alopresto@apache.org>
> > wrote:
> > >
> > >> My +1 vote from the other day is now binding.
> > >>
> > >> Andy LoPresto
> > >> alopresto@apache.org
> > >> *alopresto.apache@gmail.com <alopresto.apache@gmail.com>*
> > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >>
> > >> On Jul 8, 2016, at 9:12 AM, Joe Witt <joe.witt@gmail.com> wrote:
> > >>
> > >> +1 binding.
> > >>
> > >> Ran through full build verification.  Check licensing/notice items and
> > >> hoping we can make that leaner and meaner over time as well.
> > >>
> > >> Was able to use the toolkit to convert a nifi template to a minifi
> > >> config and fire that up and immediately see desired results.
> > >>
> > >> Some feedback items.
> > >>
> > >> 1) minifi-toolkit awareness
> > >> The build helper email references this which makes knowledge of it
> > >> available.  But I didn't initially notice any references to it in the
> > >> system admin guide doc or README entries.  Might need to call it out
> > >> more or provide a more prescriptive pointer to how to build it, find
> > >> it, use it.
> > >>
> > >> 2) minifi-toolkit execution
> > >>
> > >> When running config.sh it says
> > >> "
> > >> Transform Usage:
> > >>
> > >> java org.apache.nifi.minifi.toolkit.configuration.ConfigMain transform
> > >> INPUT_FILE OUTPUT_FILE
> > >> "
> > >>
> > >> But I ended up just running 'bin/config.sh IN.xml OUT.yml' and it
> > >> worked great so the java classname bits there are probably superfluous
> > >> or only apply to those running without the script.
> > >>
> > >> 3) WARN in logs.  Probably can be pulled in differently to avoid this.
> > >> 2016-07-08 09:04:30,300 WARN [main] org.apache.nifi.nar.NarClassLoader
> > >> ./work/nar/framework/minifi-framework-nar-0.0.1.nar-unpacked does not
> > >> contain META-INF/bundled-dependencies!
> > >>
> > >> 4) Some of these things about how minifi starts up, is configured, API
> > >> items, are probably still subject to change.  We should document that
> > >> well.  A version of 0.0.1 might not be sufficient to communicate that.
> > >>
> > >> On Fri, Jul 8, 2016 at 7:56 AM, Bryan Rosander <
> bryanrosander@gmail.com
> > >
> > >> wrote:
> > >>
> > >> +1 (non-binding)
> > >>
> > >> Downloaded RC, verified checksums, gpg.
> > >>
> > >> Built with contrib-check turned on.
> > >>
> > >> Ran through all templates at
> > >>
> > >>
> >
> https://github.com/brosander/minifi-testkit/tree/8a8b9791a6b3d6404d6eaa6be48951851d3fa898/templates
> > >> (ran config.sh transform on each template to generate yaml, ran MiNiFi
> > with
> > >> resulting yaml)
> > >>
> > >> Thanks,
> > >> Bryan
> > >>
> > >> On Fri, Jul 8, 2016 at 10:16 AM, Joe Percivall <
> > >> joepercivall@yahoo.com.invalid> wrote:
> > >>
> > >> +1 (binding)
> > >>
> > >> Ran through the helper and verified functionality.
> > >>
> > >> - - - - - -
> > >> Joseph Percivall
> > >> linkedin.com/in/Percivall
> > >> e: joepercivall@yahoo.com
> > >>
> > >>
> > >>
> > >>
> > >> On Thursday, July 7, 2016 2:13 PM, Joe Percivall
> > >> <joepercivall@yahoo.com.INVALID> wrote:
> > >> Vote forth-coming but regarding Andy's notes/questions:
> > >> I agree with using HTTPS links when possible in both files and the
> > couple
> > >> README.md changes.
> > >>
> > >> I don't think there are any plans currently for separate mailing
> > lists/IRC
> > >> nodes. I am in favor of keeping NiFi and MiNiFi together for now to
> > build
> > >> visibility, especially since MiNiFi is a sub-project of NiFi. Once
> > there is
> > >> a steady stream of questions we can revisit it.
> > >> If this vote passes, it would be the first release. In order to avoid
> > the
> > >> issues with the sensitive property you suggest would it need to be
> done
> > for
> > >> this release?
> > >> Technically Connections are referring to the IDs of both the Processor
> > and
> > >> Port. The difference is that the Processor's IDs are set as the same
> as
> > the
> > >> configured name and for the Ports, the IDs and names are those that
> are
> > set
> > >> in the target NiFi instance (you're point to a component in another
> > >> system). As you bring up this adds the the limitation that all names
> > must
> > >> be unique (which in small deployments that MiNiFi targets shouldn't
> be a
> > >> problem) but makes it more user friendly. Enforcing uniqueness was
> > added as
> > >> part of a contribution from Bryan Rosander as a validation check when
> > >> transforming the config.yml and applies to processors, connections,
> and
> > >> remote processing groups.
> > >> Joe
> > >> - - - - - - Joseph Percivalllinkedin.com/in/Percivalle:
> > >> joepercivall@yahoo.com
> > >>
> > >>
> > >>
> > >>   On Thursday, July 7, 2016 12:05 AM, Andy LoPresto <
> > >> alopresto@apache.org> wrote:
> > >>
> > >>
> > >> Hi,
> > >> I verified the GPG signature and all hash signatures. Contrib check
> ran
> > >> fine.
> > >> I used one of the example config.yml files in the System Admin Guide
> and
> > >> deployed it to the compiled binary. The tool ran successfully and
> wrote
> > to
> > >> a log file on my system. I have a couple notes about the release but
> > >> nothing blocking.
> > >> +1 (non-binding)
> > >> Notes/questions:
> > >> README.md:* Could we use HTTPS links when possible?* The note about
> the
> > >> time to run a parallel Maven build says on a moderate dev laptop it
> > should
> > >> complete in less than 10 minutes. While true, a more accurate
> statement
> > >> would be < 2 minutes. * The mkdir example references
> > “example-nifi-deploy”
> > >> and “example-minifi-deploy” interchangeably.  * Are there separate
> > MiNiFi
> > >> mailing lists and IRC nodes coming?
> > >> System Admin Guide:* Could we use HTTPS links when possible?
> > >> config.yml:* Could we please change the default algorithm for
> protecting
> > >> sensitive property values to something stronger than the current
> > selection?
> > >> I would open a Jira if necessary, but this is one of those things that
> > is
> > >> really better to do before the first release so users have a
> > >> backward-compatible config.yml file moving forward. If we change it
> in a
> > >> subsequent release, we will need to do significant migration
> > hand-holding.
> > >> My suggestion would be "PBEWITHSHA256AND256BITAES-CBC-BC” which is
> > >> significantly stronger, but after trying a few BC options, I continue
> to
> > >> get EncryptionExceptions even though I have the JCE unlimited
> > cryptographic
> > >> strength jurisdiction policy files installed, so this may be a 0.0.2
> > fix.
> > >> Is BouncyCastle not enabled by default?* Question about how processors
> > and
> > >> connectors reference each other — it appears connectors reference
> source
> > >> processors by name but destination processors or ports by ID? Is this
> > >> consistent or just demonstrating the flexibility of the system? Also,
> > what
> > >> enforces processor name uniqueness, as this is not an existing feature
> > of
> > >> NiFi.
> > >> All in all looks like a lot of great work and will expand the user
> base
> > >> and opportunities for NiFi significantly. Thanks.
> > >> Andy LoPrestoalopresto@apache.orgalopresto.apache@gmail.comPGP
> > >> Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > >>
> > >> On Jul 6, 2016, at 7:27 PM, Aldrin Piri <aldrinpiri@gmail.com> wrote:
> > >> Hello Apache NiFi Community,
> > >>
> > >> I am pleased to be calling this vote for the source release of Apache
> > NiFi
> > >> - MiNiFi,
> > >> minifi-0.0.1.
> > >>
> > >> The source zip, including signatures, digests, etc. can be found at:
> > >> https://repository.apache.org/content/repositories/orgapachenifi-1087
> > >>
> > >> The Git tag is minifi-0.0.1-RC1
> > >>
> > >> The Git commit hash is d6ebeda6bce8259f0531e8455965625333315bdb
> > >> *
> > >>
> > >>
> > >>
> >
> https://git-wip-us.apache.org/repos/asf?p=nifi-minifi.git;a=commit;h=d6ebeda6bce8259f0531e8455965625333315bdb
> > >>
> > >> *
> > >>
> > >>
> > >>
> >
> https://github.com/apache/nifi-minifi/commit/d6ebeda6bce8259f0531e8455965625333315bdb
> > >>
> > >> Checksums of minifi-0.0.1-source-release.zip:
> > >> MD5: b63355fb419ca09b26fe12485b3bf4a5
> > >> SHA1: 4c9352b7d4d53d139a707dbdcd03df73dbfb847d
> > >> SHA256:
> d52e7f614afe6aee05e6796e429f2813b51f36d2b6d83a4c08a1fb1fdb50b2ff
> > >>
> > >> Release artifacts are signed with the following key:
> > >> https://people.apache.org/keys/committer/aldrin
> > >>
> > >> KEYS file available here:
> > >> https://dist.apache.org/repos/dist/release/nifi/KEYS
> > >>
> > >> 30 issues were closed/resolved for this release:
> > >>
> > >>
> > >>
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020&version=12335481
> > >>
> > >> Release note highlights can be found here:
> > >>
> > >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/MINIFI/Release+Notes#ReleaseNotes-Version0.0.1
> > >>
> > >> The vote will be open for 72 hours.
> > >> Please download the release candidate and evaluate the necessary items
> > >> including checking hashes, signatures, build from source, and test.
> Then
> > >> please vote:
> > >>
> > >> [ ] +1 Release this package as minifi-0.0.1
> > >> [ ] +0 no opinion
> > >> [ ] -1 Do not release this package because...
> > >>
> > >> Thanks!
> > >>
> > >>
> > >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message