nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phil H <gippyp...@gmail.com>
Subject Re: Classloader issues
Date Mon, 06 Nov 2017 18:31:01 GMT
Yep, replacing the JAR with the NAR (and then re-connecting it through the UI) worked.  As
a nice postscript to the story, the previously untested filtering algorithm i had written
worked as expected first time too :-)

Thanks so much for your help, team!

> On 7 Nov 2017, at 5:22 am, Joe Witt <joe.witt@gmail.com> wrote:
> 
> Phil - you are the first person to ever make that mistake.
> 
> Nope, not true :)  No worries and thanks for following up.  I believe
> once we have the extension registry in play this sort of thing will be
> much easier/simpler.
> 
> Thanks
> 
> On Mon, Nov 6, 2017 at 1:20 PM, Phil H <gippyphil@gmail.com> wrote:
>> Hey, now I see the problem - I had (somehow?!?) managed to copy the JAR file rather
than the NAR file
>> 
>> 
>> 
>>> On 7 Nov 2017, at 5:17 am, Phil H <gippyphil@gmail.com> wrote:
>>> 
>>> Thanks Bryan,
>>> 
>>> This new NAR does not appear in the extensions directory (my other working ones
do).
>>> 
>>> As for your second question
>>> 
>>> [phil@localhost JSONCondenser]$ ls  ~/nifi-1.3.0/lib/ | grep jar
>>> javax.servlet-api-3.1.0.jar
>>> jcl-over-slf4j-1.7.25.jar
>>> jetty-schemas-3.1.jar
>>> jul-to-slf4j-1.7.25.jar
>>> log4j-over-slf4j-1.7.25.jar
>>> logback-classic-1.2.3.jar
>>> logback-core-1.2.3.jar
>>> nifi-api-1.3.0.jar
>>> nifi-framework-api-1.3.0.jar
>>> nifi-JSONCondenser-processors-0.1.jar
>>> nifi-nar-utils-1.3.0.jar
>>> nifi-properties-1.3.0.jar
>>> nifi-runtime-1.3.0.jar
>>> slf4j-api-1.7.25.jar
>>> 
>>> 
>>>> On 7 Nov 2017, at 5:13 am, Bryan Bende <bbende@gmail.com> wrote:
>>>> 
>>>> Thanks for the poms.
>>>> 
>>>> Can you provide the output of listing
>>>> NIFI_HOME/work/nar/extensions/<your-nar>.nar-unpacked/META-INF/bundled-dependencies/
>>>> ?
>>>> 
>>>> and also the output of listing JARs that are in NiFi's lib directory?
>>>> ls -l NIFI_HOME/lib/ | grep jar
>>>> 
>>>> Want to verify that nifi-utils JAR is actually in your NAR and also
>>>> that no other unexpected JARs are in your lib directory.
>>>> 
>>>> Thanks,
>>>> 
>>>> Bryan
>>>> 
>>>> On Mon, Nov 6, 2017 at 12:48 PM, Phil H <gippyphil@gmail.com> wrote:
>>>>> ./pom.xml (Note that this issue occurred BEFORE I added the external
JAR dependency, and I have that exact same org.json dependency in a bunch of other processors
I have written without issue)
>>>>> 
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>>  <modelVersion>4.0.0</modelVersion>
>>>>> 
>>>>>  <parent>
>>>>>      <groupId>org.apache.nifi</groupId>
>>>>>      <artifactId>nifi-nar-bundles</artifactId>
>>>>>      <version>1.3.0</version>
>>>>>  </parent>
>>>>> 
>>>>>  <repositories>
>>>>>      <repository>
>>>>>          <id>project.local</id>
>>>>>          <name>projects</name>
>>>>>          <url>file:${project.basedir}/repo</url>
>>>>>      </repository>
>>>>>  </repositories>
>>>>> 
>>>>>  <groupId>com.jidmu</groupId>
>>>>>  <artifactId>JSONCondenser</artifactId>
>>>>>  <version>0.1</version>
>>>>>  <packaging>pom</packaging>
>>>>> 
>>>>>  <modules>
>>>>>      <module>nifi-JSONCondenser-processors</module>
>>>>>      <module>nifi-JSONCondenser-nar</module>
>>>>>  </modules>
>>>>> 
>>>>>  <dependencies>
>>>>>      <dependency>
>>>>>          <groupId>org.json</groupId>
>>>>>          <artifactId>JSON</artifactId>
>>>>>          <version>1.0</version>
>>>>>      </dependency>
>>>>>  </dependencies>
>>>>> 
>>>>> </project>
>>>>> 
>>>>> 
>>>>> 
>>>>> ./nifi-JSONCondenser-processors/pom.xml
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>>  <modelVersion>4.0.0</modelVersion>
>>>>> 
>>>>>  <parent>
>>>>>      <groupId>com.jidmu</groupId>
>>>>>      <artifactId>JSONCondenser</artifactId>
>>>>>      <version>0.1</version>
>>>>>  </parent>
>>>>> 
>>>>>  <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>>  <packaging>jar</packaging>
>>>>> 
>>>>>  <dependencies>
>>>>>      <dependency>
>>>>>          <groupId>org.apache.nifi</groupId>
>>>>>          <artifactId>nifi-api</artifactId>
>>>>>      </dependency>
>>>>>      <dependency>
>>>>>          <groupId>org.apache.nifi</groupId>
>>>>>          <artifactId>nifi-utils</artifactId>
>>>>>      </dependency>
>>>>>      <dependency>
>>>>>          <groupId>org.apache.nifi</groupId>
>>>>>          <artifactId>nifi-mock</artifactId>
>>>>>          <scope>test</scope>
>>>>>      </dependency>
>>>>>      <dependency>
>>>>>          <groupId>org.slf4j</groupId>
>>>>>          <artifactId>slf4j-simple</artifactId>
>>>>>          <scope>test</scope>
>>>>>      </dependency>
>>>>>      <dependency>
>>>>>          <groupId>junit</groupId>
>>>>>          <artifactId>junit</artifactId>
>>>>>          <scope>test</scope>
>>>>>      </dependency>
>>>>>  </dependencies>
>>>>> </project>
>>>>> 
>>>>> ./nifi-JSONCondenser-nar/pom.xml
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>>  <modelVersion>4.0.0</modelVersion>
>>>>> 
>>>>>  <parent>
>>>>>      <groupId>com.jidmu</groupId>
>>>>>      <artifactId>JSONCondenser</artifactId>
>>>>>      <version>0.1</version>
>>>>>  </parent>
>>>>> 
>>>>>  <artifactId>nifi-JSONCondenser-nar</artifactId>
>>>>>  <version>0.1</version>
>>>>>  <packaging>nar</packaging>
>>>>>  <properties>
>>>>>      <maven.javadoc.skip>true</maven.javadoc.skip>
>>>>>      <source.skip>true</source.skip>
>>>>>  </properties>
>>>>> 
>>>>>  <dependencies>
>>>>>      <dependency>
>>>>>          <groupId>com.jidmu</groupId>
>>>>>          <artifactId>nifi-JSONCondenser-processors</artifactId>
>>>>>          <version>0.1</version>
>>>>>      </dependency>
>>>>>  </dependencies>
>>>>> 
>>>>> </project>
>>>>> 
>>>>> 
>>>>> 
>>>>>> On 7 Nov 2017, at 4:39 am, Bryan Bende <bbende@gmail.com> wrote:
>>>>>> 
>>>>>> It is most likely an issue with the Maven configuration in one of
your modules.
>>>>>> 
>>>>>> Can you share your project, or the pom files for the processors,
NAR,
>>>>>> and bundle?
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>> Bryan
>>>>>> 
>>>>>> 
>>>>>> On Mon, Nov 6, 2017 at 12:20 PM, Phil H <gippyphil@gmail.com>
wrote:
>>>>>>> Nifi version is 1.3.0, running on Java 1.8.0_131, running on
a out-of-the-box CentOS (if that’s relevant)
>>>>>>> 
>>>>>>>> On 7 Nov 2017, at 4:17 am, Phil H <gippyphil@gmail.com>
wrote:
>>>>>>>> 
>>>>>>>> I added the StandardValidators reference back in and the
original error reoccurs.  The offending code (which compiles fine using the same JDK) is totally
standard stuff:
>>>>>>>> 
>>>>>>>> public static final PropertyDescriptor ID_PATH = new PropertyDescriptor
>>>>>>>>        .Builder().name("ID_PATH")
>>>>>>>>        .displayName("JSON ID Path")
>>>>>>>>        .description("The path to the JSON attribute that
represents the unique ID for the object")
>>>>>>>>        .addValidator(StandardValidators.NON_BLANK_VALIDATOR)
>>>>>>>>        .required(true)
>>>>>>>>        .build();
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 2017-11-07 04:13:46,855 ERROR [main] org.apache.nifi.NiFi
Failure to launch NiFi due to java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor:
Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>>> java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor:
Provider com.jidmu.processors.JSONCondenser.JSONCondenser could not be instantiated
>>>>>>>>   at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>>>>>>>>   at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>>>>>>>>   at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>>>>>>>>   at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>>>>>>>>   at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>>>>>>>>   at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:138)
>>>>>>>>   at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:104)
>>>>>>>>   at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:699)
>>>>>>>>   at org.apache.nifi.NiFi.<init>(NiFi.java:160)
>>>>>>>>   at org.apache.nifi.NiFi.main(NiFi.java:267)
>>>>>>>> Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/processor/util/StandardValidators
>>>>>>>>   at com.jidmu.processors.JSONCondenser.JSONCondenser.<clinit>(JSONCondenser.java:58)
>>>>>>>>   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
>>>>>>>>   at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
>>>>>>>>   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
>>>>>>>>   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>>>>>>>   at java.lang.Class.newInstance(Class.java:442)
>>>>>>>>   at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>>>>>>>>   ... 7 common frames omitted
>>>>>>>> Caused by: java.lang.ClassNotFoundException: org.apache.nifi.processor.util.StandardValidators
>>>>>>>>   at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>>>>   at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>>>>   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
>>>>>>>>   at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>>>>   ... 14 common frames omitted
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On 7 Nov 2017, at 4:02 am, Joe Witt <joe.witt@gmail.com>
wrote:
>>>>>>>>> 
>>>>>>>>> Can you share the code by chance for a review?  Otherwise,
you'll want
>>>>>>>>> to add a debugger at runtime and examine the context
classloader as it
>>>>>>>>> goes through to see where it goes wonky.
>>>>>>>>> 
>>>>>>>>> What version are you on?
>>>>>>>>> 
>>>>>>>>> Thanks
>>>>>>>>> 
>>>>>>>>> On Mon, Nov 6, 2017 at 11:58 AM, Phil H <gippyphil@gmail.com>
wrote:
>>>>>>>>>> Hi guys,
>>>>>>>>>> 
>>>>>>>>>> I have just (today) started having issues with a
new processor I've written where a seemingly random class (e.g.: StandardValidators) fails
to load when NiFi is initializing. I've created this processor like all my others (using the
maven archetype) and the VM I'm running it on has not changed versions of any software.
>>>>>>>>>> 
>>>>>>>>>> I removed any reference to StandardValidators, and
the NAR then loaded successfully. As I added some more functionality, I then had the same
ClassNotFoundException occur with a different random class.
>>>>>>>>>> 
>>>>>>>>>> Because I have used the maven archetype, all my other
processors obviously use StandardValidators, so it's not like the class itself is somehow
corrupted. This feels like either a NiFi or maybe even JVM bug (or other issue that is surfacing
as this ClassNotFoundException).
>>>>>>>>>> 
>>>>>>>>>> Pulling my hair out here - would love some help!
>>>>>>>>>> 
>>>>>>>>>> Cheers,
>>>>>>>>>> Phil
>>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> 


Mime
View raw message