nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitry Goldenberg <dgoldenberg...@gmail.com>
Subject Re: NiFi: command-line interface ?
Date Sun, 20 Mar 2016 19:54:57 GMT
Thanks, Matt!  Let me know if there's something I can fish for in the logs.
- Dmitry

On Sun, Mar 20, 2016 at 3:49 PM, Matt Burgess <mattyb149@gmail.com> wrote:

> Hmm looks like it is working properly, not sure why you're getting the 409
> Conflict. I will look into it more.
>
> I also wanted to mention that you can make use of the nifi-client "all"
> JAR outside of Groovy shell. If you run "gradle shadowJar it will create a
> jar at build/libs/nifi-client-<version>-all.jar that you can put on a Java
> or Groovy classpath:
>
> groovy -cp build/libs/nifi-client-0.3-all.jar yourScript.groovy
>
> Then it isn't interactive, you can write your whole deployment script(s)
> using commands like you'd issue in the shell. Don't forget to import
> nifi.client.* and whatever other packages you want. The gradle shell task
> does the import nifi.client.* for you.
>
> Regards,
> Matt
>
> On Mar 20, 2016, at 3:35 PM, Dmitry Goldenberg <dgoldenberg123@gmail.com>
> wrote:
>
> Thanks, Matt,
>
> I'll take a peek at nifi-api-deploy.
>
> Meanwhile, to answer your questions:
>
>    1. nifi.currentVersion = 713
>    2. /nifi-api/controller/revision output:
>       -
>       <entity><revision><clientId>7eb95df3-ba07-4889-9b18-8d9309965a95</clientId><lastModifier>anonymous</lastModifier><version>713</version></revision></entity>
>       3. NiFi version: 0.5.1
>    4. NiFi client: git-clone of the latest, locally built with 'gradle
>    build'. Groovy Shell (2.4.5, JVM: 1.8.0_51).   I've got Gradle 2.3 whose
>    version option's output states Groovy at 2.3.9.
>
> - Dmitry
>
>
>
> On Sun, Mar 20, 2016 at 3:23 PM, Matt Burgess <mattyb149@gmail.com> wrote:
>
>> Dmitry,
>>
>> With regards to nifi-client (I am the author), that exception occurs when
>> the flow has been changed externally and the shell has not recognized it.
>> What the result of the following command?
>>
>> nifi.currentVersion
>>
>> If it is -1, then I recommend restarting the shell. It should be a
>> non-negative integer, a -1 usually indicates there was an error
>> communicating with the NiFi instance, or it couldn't parse the response.
>> The response containing the current revision of the NiFi workflow can be
>> obtained from your browser or "curl" using the following REST endpoint:
>>
>> /nifi-api/controller/revision
>>
>> It should be a JSON document, but there isn't any code in nifi-client
>> currently to force a JSON response. I will add that shortly for version 0.4
>> (which will allow you to add processors, interact with process groups,
>> etc.)  What version of nifi-client and NiFi are you using?
>>
>> nifi-client is a Groovy object model for NiFi concepts (processor, e.g.),
>> and performing operations on the objects results in calls to the NiFi REST
>> API (https://nifi.apache.org/docs/nifi-docs/rest-api/index.html) To
>> answer the questions:
>>
>> 1) Your nifi-client example code should work as-is (but see above)
>>
>> 2) There is at least one other solution for command-line deployment:
>> https://github.com/aperepel/nifi-api-deploy.  It uses Groovy scripts and
>> YAML configuration files to interact with the REST API.  These are
>> contributions from the community and not "official" command-line tools for
>> Apache NiFi. Having said that, I'm quite sure we'd both appreciate any
>> suggestions (and especially contributions!) to make these better.
>>
>> The "official" API is the REST API (see link above), so you can use
>> "curl" or something like that to interact with the NiFi instance. However
>> Andrew and I have both used Groovy to abstract the complexities of the API
>> away from the user.
>>
>> Regards,
>> Matt
>>
>> On Sun, Mar 20, 2016 at 2:37 PM, Dmitry Goldenberg <
>> dgoldenberg123@gmail.com> wrote:
>>
>>> I see some posts on the Web regarding NiFi's command-line interface.
>>> One of them lead me to a client library
>>> <https://github.com/mattyb149/nifi-client> that one can use via a
>>> Groovy shell.
>>>
>>> I seem to have been able to instantiate my sample dataflow through this
>>> interface but when I tried kicking off the dataflow by starting its first
>>> processor, I got an exception "processor:[id:6c71c1c9-1220-44a2-a020-8bd3c82d1692,
>>> state:RUNNING]] groovyx.net.http.HttpResponseException: Conflict.
>>>
>>> In the canvas, I've made sure nothing is running but I'm getting this
>>> exception (full stack trace below).
>>>
>>> My questions are:
>>>
>>> a) is this the right way to kick off a given existing dataflow;
>>> b) are there any other command-line options for performing operations on
>>> dataflows (and templates)?
>>>
>>> The CLI options are often critical from the DevOps perspective, mostly
>>> for being able to script things up.
>>>
>>> Thanks.
>>> - Dmitry
>>>
>>> ---------------------------------------
>>>
>>>
>>> *nifi-shell:*000*>* nifi.processors.'GetXmlFiles'.start()
>>>
>>> [revision:[version:713],
>>> processor:[id:6c71c1c9-1220-44a2-a020-8bd3c82d1692, state:RUNNING]]
>>>
>>> groovyx.net.http.HttpResponseException: Conflict
>>>
>>> at
>>> groovyx.net.http.HTTPBuilder.defaultFailureHandler(HTTPBuilder.java:651)
>>>
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:497)
>>>
>>> at
>>> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
>>>
>>> at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
>>>
>>> at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1210)
>>>
>>> at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1077)
>>>
>>> at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1019)
>>>
>>> at groovy.lang.Closure.call(Closure.java:426)
>>>
>>> at groovyx.net.http.HTTPBuilder$1.handleResponse(HTTPBuilder.java:503)
>>>
>>> at
>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:1070)
>>>
>>> at
>>> org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:1044)
>>>
>>> at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:515)
>>>
>>> at groovyx.net.http.HTTPBuilder.doRequest(HTTPBuilder.java:434)
>>>
>>> at groovyx.net.http.HTTPBuilder.request(HTTPBuilder.java:366)
>>>
>>> at groovyx.net.http.HTTPBuilder$request.call(Unknown Source)
>>>
>>> at nifi.client.Processor.updateState(Processor.groovy:149)
>>>
>>> at nifi.client.Processor$updateState$0.callCurrent(Unknown Source)
>>>
>>> at nifi.client.Processor.start(Processor.groovy:127)
>>>
>>> at nifi.client.Processor$start.call(Unknown Source)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
>>>
>>> at groovysh_evaluate.run(groovysh_evaluate:3)
>>>
>>> at groovysh_evaluate$run.call(Unknown Source)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
>>>
>>> at groovysh_evaluate$run.call(Unknown Source)
>>>
>>> at
>>> org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:70)
>>>
>>> at org.codehaus.groovy.tools.shell.Evaluator$evaluate.call(Unknown
>>> Source)
>>>
>>> at org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:187)
>>>
>>> at org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:122)
>>>
>>> at org.codehaus.groovy.tools.shell.Shell$leftShift$0.call(Unknown Source)
>>>
>>> at
>>> org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:95)
>>>
>>> at
>>> org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
>>>
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:497)
>>>
>>> at
>>> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
>>>
>>> at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
>>>
>>> at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1210)
>>>
>>> at
>>> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
>>>
>>> at
>>> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
>>>
>>> at
>>> org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124)
>>>
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:497)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:158)
>>>
>>> at org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:59)
>>>
>>> at
>>> org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
>>>
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:497)
>>>
>>> at
>>> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
>>>
>>> at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
>>>
>>> at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1210)
>>>
>>> at
>>> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
>>>
>>> at
>>> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
>>>
>>> at
>>> org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:83)
>>>
>>> at java_lang_Runnable$run.call(Unknown Source)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
>>>
>>> at org.codehaus.groovy.tools.shell.Groovysh.run(Groovysh.groovy:587)
>>>
>>> at org.codehaus.groovy.tools.shell.Groovysh$run$0.callCurrent(Unknown
>>> Source)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166)
>>>
>>> at org.codehaus.groovy.tools.shell.Groovysh.run(Groovysh.groovy:552)
>>>
>>> at org.codehaus.groovy.tools.shell.Groovysh$run.call(Unknown Source)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
>>>
>>> at org.codehaus.groovy.tools.shell.Main.startGroovysh(Main.groovy:186)
>>>
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:497)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
>>>
>>> at
>>> org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
>>>
>>> at org.codehaus.groovy.tools.shell.Main.main(Main.groovy:152)
>>>
>>>
>>>
>>
>

Mime
View raw message