nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron Rich <aaron.r...@gmail.com>
Subject Re: CM of nar via toolkit cli and regisry
Date Mon, 22 Jul 2019 18:04:08 GMT
Hi Jeff,
Thanks for the information on the release process.

For the error I'm getting, I started a new custom service with:
mvn archetype:generate -DarchetypeGroupId=org.apache.nifi
-DarchetypeArtifactId=nifi-service-bundle-archetype
-DarchetypeVersion=1.9.2 -DnifiVersion=1.9.2

This provides a shell test which I haven't updated.

I updated the testservice-api-nar and the  testservice-nar poms like I did
with the processor to include:
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-api</artifactId>
<version>1.10.0-SNAPSHOT</version>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-nar-maven-plugin</artifactId>
<version>1.3.1</version>
<extensions>true</extensions>
</plugin>
</plugins>
</build>

On the testService-api-nar I still get the warning of:
[INFO] Generating documentation for NiFi extensions in the NAR...
[INFO] Found NAR dependency of
com.test.nifi:nifi-testService-api-nar:nar:1.0-SNAPSHOT:compile
[INFO] Found NAR dependency of
org.apache.nifi:nifi-standard-services-api-nar:nar:1.9.2:compile
[INFO] Found NAR dependency of
org.apache.nifi:nifi-jetty-bundle:nar:1.9.2:compile
[INFO] Found a dependency on version 1.9.2 of NiFi API
[WARNING] Could not generate extensions' documentation
org.apache.maven.plugin.MojoExecutionException: Failed to create Extension
Documentation

When I was trying to trace back the 1.9.2 NiFi API dependency, I changed
the testService to have nifi-api version 1.10. This is when the compile
errors showed up.

So at this point my question would be, for a custom service based on 1.9.2,
which pom.xml files need to be changed, and with what changes, to have the
service.nar file have the documentation to be able to be uploaded to nifi
registry via cli? Seems like there is more then just moving nifi-api to
version 1.10.

Thank you again for the help.

-Aaron



On Mon, Jul 22, 2019 at 12:43 AM Jeff <jtswork@gmail.com> wrote:

> Aaron,
>
> New versions of NiFi are generally released after a member of the community
> proposes a release and someone volunteers to perform the duties of the
> release manager.  There is no predefined schedule or projected release
> dates.  I think there are a few major feature JIRAs on which work is
> currently being done, and once those are resolved/finished, at that point
> someone may make proposal to create a release candidate for 1.10.0.
>
> Regarding the error to which you are referring, does the
> TestStandardMyService class make use of nifi-mock classes?  Is
> TestStandardMyService within the "src/main" folder of your module?  If so,
> you may want to move that class into "src/test", as I don't believe the
> nifi-maven-nar plugin bundles test code when it creates the NAR.  From the
> stacktrace, I see that TestStandardMyService makes a call to a
> org.apache.nifi.util.MockControllerServiceIntitializationContext method,
> which comes from nifi-mock.  If TestStandardMyService is included with the
> rest of your "production" code, you might want to refactor your module so
> that it is part of the "test" code.  That should remove the dependency on
> nifi-mock.
>
> On Mon, Jul 22, 2019 at 12:24 AM Aaron Rich <aaron.rich@gmail.com> wrote:
>
> > I updated  pom  for  a bundle with just a processor and was able to get
> nar
> > generated and loaded into registry via cli.
> >
> > When I tried a bundle with a custom service though, I get failures in the
> > test:
> > java.lang.AbstractMethodError:
> >
> >
> org.apache.nifi.util.MockControllerServiceInitializationContext.getNodeTypeProvider()Lorg/apache/nifi/controller/NodeTypeProvider;
> >         at
> >
> >
> com.ngc.swordfish.nifi.multipart.TestStandardMyService.testService(TestStandardMyService.java:36)
> >
> > Do I need to updated nifi-mock to 1.10 also?
> >
> > Thanks.
> >
> > -Aaron
> >
> > On Sun, Jul 21, 2019 at 9:59 PM Aaron Rich <aaron.rich@gmail.com> wrote:
> >
> > > Also,
> > >
> > > When is 1.10.0 planned for release?
> > >
> > > Thanks.
> > >
> > > -Aaron
> > >
> > >
> > > On Sun, Jul 21, 2019 at 9:29 PM Aaron Rich <aaron.rich@gmail.com>
> wrote:
> > >
> > >> Great.
> > >>
> > >> Thank you.
> > >>
> > >> Is the best way to force my nar to use the 1.10 nifi-api via adding it
> > as
> > >> dependency is main pom file?
> > >>
> > >> Assuming I still need the build plugin also?
> > >>
> > >> Thanks again.
> > >>
> > >> -Aaron
> > >>
> > >> On Sun, Jul 21, 2019 at 9:30 AM Bryan Bende <bbende@gmail.com> wrote:
> > >>
> > >>> Hello,
> > >>>
> > >>> The issue is that in order to correctly generate the extension
> manifest
> > >>> with the new NAR plugin, it requires changes from nifi-api that are
> not
> > >>> released yet.
> > >>>
> > >>> You should be able to build NiFi on the master branch, really just
> the
> > >>> nifi-api module, doing a mvn clean install.
> > >>>
> > >>> Then in your NAR you’ll need to force it to use 1.10.0-SNAPSHOT of
> > >>> nifi-api. Currently you are getting 1.9.2 because of the parent of
> > >>> nifi-nar-bundles.
> > >>>
> > >>> Once we get 1.10.0 released then this won’t be an issue anymore.
> > >>>
> > >>> Thanks,
> > >>>
> > >>> Bryan
> > >>>
> > >>> On Sat, Jul 20, 2019 at 11:36 PM Aaron Rich <aaron.rich@gmail.com>
> > >>> wrote:
> > >>>
> > >>> > I forgot to include that when generating the .nar, I do get the
> > >>> warnings
> > >>> > of:
> > >>> > [WARNING] Could not generate extensions' documentation
> > >>> > org.apache.maven.plugin.MojoExecutionException: Failed to create
> > >>> Extension
> > >>> > Documentation
> > >>> > ...
> > >>> > Caused by: java.lang.NoSuchMethodException:
> > >>> >
> > >>> >
> > >>>
> >
> org.apache.nifi.documentation.xml.XmlDocumentationWriter.initialize(org.apache.nifi.components.ConfigurableComponent)
> > >>> >
> > >>> > I believe this is tied to not getting the right nfi-api version?
> But
> > >>> the
> > >>> > parent is set to 1.9.2 so not sure how that is happening.
> > >>> >
> > >>> > Thanks again.
> > >>> >
> > >>> > -Aaron
> > >>> >
> > >>> > On Sat, Jul 20, 2019 at 9:09 PM Aaron Rich <aaron.rich@gmail.com>
> > >>> wrote:
> > >>> >
> > >>> > > Hi,
> > >>> > >
> > >>> > > I'm trying to determine the best way to CM custom nar files
for
> > >>> sharing
> > >>> > > between team. We are using nifi-registry for the flows and
I saw
> > >>> there
> > >>> > was
> > >>> > > a new capability via the toolkit cli for "upload-bundle".
> > >>> > >
> > >>> > > I'm trying to use that but have ran into a few issues:
> > >>> > > 1) I first wasn't getting the META-INF/docs/ in the .nar.
I had
> > >>> built the
> > >>> > > initial project from mvn archetype:generate with version
1.9.2. I
> > >>> added:
> > >>> > > <build>
> > >>> > > <plugins>
> > >>> > > <plugin>
> > >>> > > <groupId>org.apache.nifi</groupId>
> > >>> > > <artifactId>nifi-nar-maven-plugin</artifactId>
> > >>> > > <version>1.3.1</version>
> > >>> > > <extensions>true</extensions>
> > >>> > > </plugin>
> > >>> > > </plugins>
> > >>> > > </build>
> > >>> > >
> > >>> > > To the base pom.xml. It has the parent of:
> > >>> > > <parent>
> > >>> > > <groupId>org.apache.nifi</groupId>
> > >>> > > <artifactId>nifi-nar-bundles</artifactId>
> > >>> > > <version>1.9.2</version>
> > >>> > > </parent>
> > >>> > >
> > >>> > > That got the docs in the jar:
> > >>> > >      0 Sat Jul 20 20:53:44 MDT 2019 META-INF/docs/
> > >>> > >     72 Sat Jul 20 20:53:44 MDT 2019
> > >>> META-INF/docs/extension-manifest.xml
> > >>> > >      0 Sat Jul 20 20:53:44 MDT 2019
> > META-INF/docs/additional-details/
> > >>> > >
> > >>> > > 2)I tried to use cli then with command:
> > >>> > > ./bin/cli.sh registry upload-bundle --baseUrl
> > >>> http://localhost:18080/
> > >>> > > --bucketId bcc0865e-818f-475b-b506-c95614b9a9d1 -ebt nifi-nar
> -ebf
> > >>> > > .custom-processor-nar-1.0-SNAPSHOT.nar
> > >>> > >
> > >>> > > I get back "ERROR: Error executing command 'upload-bundle'
:
> Error
> > >>> > > creating extension bundle version: An unexpected error has
> > occurred.
> > >>> > Please
> > >>> > > check the logs for additional details."
> > >>> > >
> > >>> > > In Registry log, error is:
> > >>> > > Exception: Unable to obtain extension info for bundle due
to:
> > Unable
> > >>> to
> > >>> > > parse extension manifest due to: Unexpected end of input
block;
> > >>> expected
> > >>> > an
> > >>> > > identifier
> > >>> > >  at [row,col {unknown-source}]: [1,72]
> > >>> > >  at [Source:
> > >>> > >
> > >>> >
> > >>>
> >
> (org.apache.nifi.registry.bundle.extract.nar.NarBundleExtractor$NonCloseableInputStream);
> > >>> > > line: 1, column: 73]. Returning Internal Server Error response.
> > >>> > > org.apache.nifi.registry.bundle.extract.BundleException:
Unable
> to
> > >>> obtain
> > >>> > > extension info for bundle due to: Unable to parse extension
> > manifest
> > >>> due
> > >>> > > to: Unexpected end of input
> > >>> > > block; expected an identifier
> > >>> > >
> > >>> > > Questions:
> > >>> > > 1) Should I need to set the 1.3.1 plugin do get the docs
in nar?
> I
> > >>> > thought
> > >>> > > that would be automatic from parent version being 1.9.2
> > >>> > > 2) Is there an example for how to use the toolkit cli to
> > >>> upload-bundle?
> > >>> > > 3) Is there a better way to CM .nar? Archiva? Artifactory?
> > >>> > >
> > >>> > > Thanks.
> > >>> > >
> > >>> > > -Aaron
> > >>> > >
> > >>> > >
> > >>> >
> > >>> --
> > >>> Sent from Gmail Mobile
> > >>>
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message