nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anshuman Ghosh <anshuman.ghosh2...@gmail.com>
Subject Re: Testing automation, Deployment in NiFi
Date Mon, 27 Feb 2017 16:45:49 GMT
Hello Bryan,

Greetings of the day!

Thank you very much for your reply.
I have quickly looked into the documents and we are looking forward for the
deployment stuffs that are in development now.

However it would be great, if I get some more guidance on the REST API part
for testing automation. I am very new in this area so any help would be
really beneficial.

   1. Is it like first we need to create the flow in Web UI and then create
   a script to instantiate/ start it?
   2. I have attached a DataFlow snap here (PFA), so for this what you
   would suggest? Can I create the entire Flow using REST API or how does that
   work?

I know thesse might be basics again but I do not have knowledge on this!
Thanking you in advance.


‚Äč
______________________

*Kind Regards,*
*Anshuman Ghosh*
*Contact - +49 179 9090964*


On Fri, Feb 24, 2017 at 5:35 PM, Bryan Bende <bbende@gmail.com> wrote:

> Hello,
>
> In terms of automated testing, anything that you can do in the UI can
> also be done via the REST API [1]. So you could write scripts/code to
> create a process group, then instantiate processors, connect them,
> start them, send in known data, and check the output somewhere.
>
> As far as deployment, this is an area that the community is currently
> working to improve [2]. Currently you can use templates as a
> deployment vehicle [3][4], or you can copy the whole flow.xml.gz from
> one environment to the other, assuming you want to completely move the
> whole canvas from one environment to the other (depends on sensitive
> properties encryption key being the same in each environment). This is
> definitely an area that will significantly improve in the future, and
> we are happy to hear and discuss on feedback on the feature proposals.
>
> Thanks,
>
> Bryan
>
> [1] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html
> [2] https://cwiki.apache.org/confluence/display/NIFI/
> Configuration+Management+of+Flows
> [3] https://github.com/aperepel/nifi-api-deploy
> [4] https://github.com/ijokarumawak/nifi-deploy-process-group
>
> On Fri, Feb 24, 2017 at 7:54 AM, Anshuman Ghosh
> <anshuman.ghosh2009@gmail.com> wrote:
> > Hello team,
> >
> > Greetings of the day!
> >
> > This is Anshu here and we in our organization have started with Apache
> NiFi
> > recently.
> > I must appreciate the features and ease of usage of this tool and the
> > community response, however I have couple of questions since we are in
> the
> > beginning.
> > It would really be great if we can get some guidance on this.
> >
> > *Brief about our use case. *
> > - We need to consume data from one/ more Kafka topics.
> > - Need to transform data on the fly.
> > - Need to write to HDFS finally with some partition logic.
> >
> > *What are we looking for?*
> > - Please guide us on how to automate our end to end testing.
> > - How to handle deployment in NiFi. If we develop and test a flow in
> lower
> > environments, is there a way to package them together and deploy them in
> > higher environment?
> >
> > Thanking you in advance!
> > Should there be any information required from my side, please do let me
> > know.
> >
> >
> >
> > ______________________
> >
> > *Kind Regards,*
> > *Anshuman Ghosh*
> > *Contact - +49 179 9090964*
>

Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message