nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Payne <marka...@hotmail.com>
Subject Re: Adding JVM arguments when running NiFi...
Date Wed, 18 Jan 2017 15:31:39 GMT
Russ,

Sorry - I didn't see this sooner. That is accurate, that 'N' can be non-numeric. By default,
in bootstrap.conf, there is actually a property named "java.arg.debug=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000"
but it is commented out.

I think that if we want to guarantee ordering, we need to do some sort of sorting on the keys.
For example, first comes all numbers in numeric ordering (i.e., - not lexigraphical ordering
- so java.arg.1, java.arg.2, java.arg.3, ..., java.arg.10, java.arg.11, ...), followed by
non-numeric values in lexigraphical ordering. This would make it much easier to guarantee
ordering by simply using numbers but doesn't enforce numbers to be used. We didn't enforce
numbers to be used because historically we've found that people are inclined to always use
a one-up number. So when they want to add a new property, they go searching through the file
looking for the latest number used and then use the next number; then, if they un-comment
a line, that one may conflict resulting in two arguments with the same numbers and one of
them is overridden.

Thanks
-Mark


> On Jan 18, 2017, at 10:22 AM, Russell Bateman <russ@windofkeltia.com> wrote:
> 
> For my own purposes, I fixed this in 0.7.1 /nifi-bootstrap///src/main/java/org/////apache/nifi/bootstrap/RunNiFi.java/,
but at Joe's suggestion, I created JIRA NIFI-3364 hoping some committer will get to it for
all code branches-forward. The solution I used was made to be as little disruptive to the
existing code as possible. To that JIRA I attached the solution pinched between comments in
a larger test demonstrating it.
> 
> I had found that I could trick the order in my local environment by trial and error,
however, moving to another host and with the additional need to dominate the order of JMX
remote definitions as well (in order to reach NiFi remote from Java Mission Control), it just
became impossible to achieve it.
> 
> The code I wrote isn't the only way to fix nor even necessarily the best, but it works
supporting both the ordering of /N/in java.arg.Nas well as (did you know this?) the use of
non-numeric options for the position of /N/.
> 
> Russ
> 
> On 12/14/2016 11:05 AM, Russell Bateman wrote:
>> 
>> I've doctored /conf/bootstrap.conf/ to contain these additional lines:
>> 
>>    java.arg.15=-XX:+UnlockCommercialFeatures
>>    java.arg.16=-XX:+FlightRecorder
>>    java.arg.17=-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>> 
>> 
>> In the end, NiFi's grumpy about this and won't start (from /logs/nifi-bootstrap.log/):
>> 
>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error:
*To use 'FlightRecorder', first unlock using -XX:+UnlockCommercialFeatures.*
>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error:
Could not create the Java Virtual Machine.
>> 2016-12-14 10:39:36,489 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error:
A fatal exception has occurred. Program will exit.
>> 2016-12-14 10:39:36,507 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never
started. Will not restart NiFi
>> 
>> I tried using all options as one (in case the order is disturbed, which it was):
>> 
>>    java.arg.15=-XX:+UnlockCommercialFeatures-XX:+FlightRecorder-XX:StartFlightRecording=duration=120m,filename=recording.jfr
>> 
>> 
>> and then I get:
>> 
>> *2016-12-14 10:50:07,574 ERROR [NiFi logging handler] org.apache.nifi.StdErr Unrecognized
VM option 'UnlockCommercialFeatures -XX:+FlightRecorder -XX:StartFlightRecording=duration=120m,filename=recording.jfr'*
>> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error:
Could not create the Java Virtual Machine.
>> 2016-12-14 10:50:07,574 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error:
A fatal exception has occurred. Program will exit.
>> 2016-12-14 10:50:07,598 INFO [main] org.apache.nifi.bootstrap.RunNiFi NiFi never
started. Will not restart NiFi
>> 
>> Here's the second command line from /logs/nifi-bootstrap.log/, which I've wrapped
for convenience in reading (see JVM version here too):
>> 
>> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command Working Directory:
/home/russ/dev/nifi/nifi-1.1.0
>> 2016-12-14 10:56:40,526 INFO [main] org.apache.nifi.bootstrap.Command Command: */home/russ/dev/jdk1.8.0_112/bin/java*
>> -classpath /home/russ/dev/nifi/nifi-1.1.0/./conf:/home/russ/dev/nifi/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/home/russ/dev/nifi/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar
>> -Dorg.apache.jasper.compiler.disablejsr199=true
>> -Xmx512m
>> -Xms512m
>> -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=9191
>> *-XX:+UnlockCommercialFeatures
>> -XX:+FlightRecorder
>> -XX:StartFlightRecording=duration=120m,filename=recording.jfr*
>> -Dsun.net.http.allowRestrictedHeaders=true
>> -Djava.net.preferIPv4Stack=true
>> -Djava.awt.headless=true
>> -XX:+UseG1GC
>> -Djava.protocol.handler.pkgs=sun.net.www.protocol
>> -Dnifi.properties.file.path=/home/russ/dev/nifi/nifi-1.1.0/./conf/nifi.properties
>> -Dnifi.bootstrap.listen.port=41736
>> -Dapp=NiFi
>> -Dorg.apache.nifi.bootstrap.config.log.dir=/home/russ/dev/nifi/nifi-1.1.0/logs org.apache.nifi.NiFi
>> 
>> Last, I frantically tried this:
>> 
>>    java.arg.15="-XX:+UnlockCommercialFeatures -XX:+FlightRecorder
>>    -XX:StartFlightRecording=duration=120m,filename=recording.jfr"
>> 
>> 
>> and then got:
>> 
>> *2016-12-14 11:01:36,659 ERROR [NiFi logging handler] org.apache.nifi.StdErr Error:
Could not find or load main class "-XX:+UnlockCommercialFeatures -XX:+FlightRecorder -XX:StartFlightRecording=duration=120m,filename=recording.jfr"*
>> 
>> 
>> What's the canonical way of adding/inserting my own JVM commands? Alternatively,
is NiFi analyzing and blocking -XX:arguments to the JVM as illegal or unrecognized?
>> 
>> Thanks,
>> 
>> Russ
> 


Mime
View raw message