nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sivaprasanna <sivaprasanna...@gmail.com>
Subject Re: GetMongoDB : How to pass parameters as input to GetMongoDB processor
Date Fri, 04 May 2018 07:30:19 GMT
As Bryan mentioned in the actual ‘GetSplunk’ thread, it is not available
but it certainly makes sense to have that feature. If you’re interested in
having that, please raise a Jira at https://issues.apache.org/jira

-
Sivaprasanna

On Fri, 4 May 2018 at 12:56 PM, Brajendra Mishra <
brajendra_mishra@persistent.com> wrote:

> Hi Mike,
>
> Thanks a lot for your valuable inputs.
> We tried GetMongoDB Template which you shared in previous mail with new
> version of Apache NiFi (NiFi-1.6) , Its working fine and we are able to get
> mongoDB data in desired format.
>
> One query: Is such input flow functionality available with 'GetSplunk'
> processor too with NiFI 1.6.0? if yes, please share sample template or if
> not, When can expect such implementation with 'GetSplunk' processor too?
>
> Brajendra Mishra
> Persistent Systems Ltd.
>
> -----Original Message-----
> From: Mike Thomsen <mikerthomsen@gmail.com>
> Sent: Thursday, May 03, 2018 6:56 PM
> To: dev@nifi.apache.org
> Subject: Re: GetMongoDB : How to pass parameters as input to GetMongoDB
> processor
>
> Brajendra,
>
> I would recommend an update to 1.6.0. It'll make your life a lot easier on
> this. I did that patch to GetMongo because I had a client that had an
> explosion of GetMongos due to that inflexibility. With that said, *be aware
> of this bug* in 1.6.0 w/ PutMongo if you use it and upgrade. It is fixed in
> 1.7.0 (still in development):
>
> Migrating from 1.5.0 to 1.6.0
>
>    - PutMongo can fail in insert mode. Will be fixed in next release. In
>    the mean time you can set query keys for insert even though they'll be
>    ignored it should workaround the validation bug.
>
>
> What it means is there is a validator function that is broken in PutMongo
> when one is using the "insert mode" instead of "update mode." You can do a
> work around by putting a dummy value in the "query key" field to make it
> happy.
>
> On Thu, May 3, 2018 at 8:26 AM Pierre Villard <pierre.villard.fr@gmail.com
> >
> wrote:
>
> > Hi,
> >
> > As Mike said: incoming relationship has been added for NiFi 1.6.0.
> > https://issues.apache.org/jira/browse/NIFI-4827
> >
> > Pierre
> >
> > 2018-05-03 14:09 GMT+02:00 Brajendra Mishra <
> > brajendra_mishra@persistent.com
> > >:
> >
> > > Hi Mike,
> > >
> > > I did attach the same in my previous mail. Well reattaching it again.
> > > Well error is at GetMongoDB Processor and error text is : "Upstream
> > > Connections is invalid because Processor does not allow upstream
> > > connections but currently has 1"
> > >
> > > Brajendra Mishra
> > > Persistent Systems Ltd.
> > >
> > > -----Original Message-----
> > > From: Mike Thomsen <mikerthomsen@gmail.com>
> > > Sent: Thursday, May 03, 2018 5:20 PM
> > > To: dev@nifi.apache.org
> > > Subject: Re: GetMongoDB : How to pass parameters as input to
> > > GetMongoDB processor
> > >
> > > Brajendra,
> > >
> > > Looks like the image didn't make it.
> > >
> > > On Wed, May 2, 2018 at 11:36 PM Brajendra Mishra <
> > > brajendra_mishra@persistent.com> wrote:
> > >
> > > > Hi Mike,
> > > >
> > > > Thanks for responding.
> > > >
> > > > Here, I have attached missing image attachment.
> > > >
> > > >
> > > >
> > > > Brajendra Mishra
> > > >
> > > > Persistent Systems Ltd.
> > > >
> > > >
> > > >
> > > > *From:* Mike Thomsen <mikerthomsen@gmail.com>
> > > > *Sent:* Wednesday, May 02, 2018 6:24 PM
> > > >
> > > >
> > > > *To:* dev@nifi.apache.org
> > > > *Subject:* Re: GetMongoDB : How to pass parameters as input to
> > > > GetMongoDB processor
> > > >
> > > >
> > > >
> > > > That might require 1.6.0. Also, your image didn't come through in
> > > > your response to Sivaprasanna so resend that too.
> > > >
> > > > On Wed, May 2, 2018 at 8:37 AM Brajendra Mishra <
> > > > brajendra_mishra@persistent.com> wrote:
> > > >
> > > > Hi Mike,
> > > >
> > > > Thanks a lot for responding.
> > > >
> > > > On your statement
> > > > "That is its new default behavior if you leave the query field
> > > > blank and have an incoming connection from another processor. That
> > > > would be a good way to integrate the flow with another application"
> > > >
> > > > Could you please share a sample template for the same?
> > > >
> > > >
> > > > Brajendra Mishra
> > > > Persistent Systems Ltd.
> > > >
> > > > -----Original Message-----
> > > > From: Mike Thomsen <mikerthomsen@gmail.com>
> > > > Sent: Wednesday, May 02, 2018 5:58 PM
> > > > To: dev@nifi.apache.org
> > > > Subject: Re: GetMongoDB : How to pass parameters as input to
> > > > GetMongoDB processor
> > > >
> > > > GetMongo can also use the body of a flowfile for the query. That
> > > > is its new default behavior if you leave the query field blank and
> > > > have an incoming connection from another processor. That would be
> > > > a good way to integrate the flow with another application. For
> > > > example, you could add FetchKafka to the flow and have your
> > > > applications post messages to Kafka with the queries they want it
> > > > to run and FetchKafka would send that JSON to GetMongo as it comes
> > > > in. Or you could build a REST service that writes the JSON to disk
> and use GetFile to load it.
> > > Lots of ways to do this.
> > > >
> > > > On Wed, May 2, 2018 at 6:42 AM Sivaprasanna
> > > > <sivaprasanna246@gmail.com>
> > > > wrote:
> > > >
> > > > > Since I'm not so sure about your exact use case, I have just
> > > > > created a rough template based on the simple example flow that I
> > > > > had posted earlier which is GenerateFlowfile -> UpdateAttribute
> > > > > -> GetMongo. I have attached the template here.
> > > > >
> > > > > -
> > > > > Sivaprasanna
> > > > >
> > > > > On Wed, May 2, 2018 at 2:55 PM, Brajendra Mishra <
> > > > > brajendra_mishra@persistent.com> wrote:
> > > > >
> > > > >> Hi Sivaprasanna,
> > > > >>
> > > > >> Could you please provide me the sample template for the same,
> > > > >> where I can pass parameters (and get those parameters' value
to
> > > > >> process
> > > > >> further) to GetMongoDB processor?
> > > > >> It would be a great help for us.
> > > > >>
> > > > >> Brajendra Mishra
> > > > >> Persistent Systems Ltd.
> > > > >>
> > > > >> -----Original Message-----
> > > > >> From: Sivaprasanna <sivaprasanna246@gmail.com>
> > > > >> Sent: Wednesday, May 02, 2018 2:28 PM
> > > > >> To: dev@nifi.apache.org
> > > > >> Subject: Re: GetMongoDB : How to pass parameters as input to
> > > > >> GetMongoDB processor
> > > > >>
> > > > >> Hi.
> > > > >>
> > > > >> GetMongo can take input. So technically you can use a processor
> > > > >> before and then connect it  to GetMongo.
> > > > >>
> > > > >> A simple example :
> > > > >> GenerateFlowfile -> UpdateAttribute -> GetMongo
> > > > >>
> > > > >> In the UpdateAttribute, you can add attributes for the database
> > > > >> and collection and then use them in GetMong using NiFi
> > > > >> Expression
> > > Language.
> > > > >>
> > > > >> Let me know, if that doesn’t help.
> > > > >>
> > > > >> -
> > > > >> Sivaprasanna
> > > > >>
> > > > >> On Wed, 2 May 2018 at 1:26 PM, Brajendra Mishra <
> > > > >> brajendra_mishra@persistent.com> wrote:
> > > > >>
> > > > >> > Hi Team,
> > > > >> > We have found there is only 'GetMongoDB' processor to connect
> > > > >> > and query to MongoDB in Apache NiFi.
> > > > >> > Hence, we this processor does not take any type or input.
> > > > >> >
> > > > >> > Do we have another type to Apache NiFi processor which can
> > > > >> > take parameters as input (details of MongoDB, query, instance
> > > > >> > etc.) from
> > > > >> other processor?
> > > > >> > If not then please suggest when such type of processor can
be
> > > > >> > expected in upcoming release?
> > > > >> >
> > > > >> > Brajendra Mishra
> > > > >> > Persistent Systems Ltd.
> > > > >> >
> > > > >> > DISCLAIMER
> > > > >> > ==========
> > > > >> > This e-mail may contain privileged and confidential
> > > > >> > information which is the property of Persistent Systems
Ltd.
> > > > >> > It is intended only for the use of the individual or entity
> > > > >> > to which it is addressed. If you are not the intended
> > > > >> > recipient, you are not authorized to read, retain, copy,
> > > > >> > print, distribute or use this message. If you have received
> > > > >> > this communication in error, please notify the sender and
> > > > >> > delete all copies
> > > > >> of this message.
> > > > >> > Persistent Systems Ltd. does not accept any liability for
> > > > >> > virus infected mails.
> > > > >> >
> > > > >> DISCLAIMER
> > > > >> ==========
> > > > >> This e-mail may contain privileged and confidential information
> > > > >> which is the property of Persistent Systems Ltd. It is intended
> > > > >> only for the use of the individual or entity to which it is
> > > > >> addressed. If you are not the intended recipient, you are not
> > > > >> authorized to read, retain, copy, print, distribute or use this
> > > > >> message. If you have received this communication in error,
> > > > >> please notify the sender and
> > > > delete all copies of this message.
> > > > >> Persistent Systems Ltd. does not accept any liability for virus
> > > > >> infected mails.
> > > > >>
> > > > >
> > > > >
> > > > DISCLAIMER
> > > > ==========
> > > > This e-mail may contain privileged and confidential information
> > > > which is the property of Persistent Systems Ltd. It is intended
> > > > only for the use of the individual or entity to which it is
> > > > addressed. If you are not the intended recipient, you are not
> > > > authorized to read, retain, copy, print, distribute or use this
> > > > message. If you have received this communication in error, please
> > > > notify the sender and delete all copies
> > > of this message.
> > > > Persistent Systems Ltd. does not accept any liability for virus
> > > > infected mails.
> > > >
> > > >
> > > DISCLAIMER
> > > ==========
> > > This e-mail may contain privileged and confidential information
> > > which is the property of Persistent Systems Ltd. It is intended only
> > > for the use
> > of
> > > the individual or entity to which it is addressed. If you are not
> > > the intended recipient, you are not authorized to read, retain,
> > > copy, print, distribute or use this message. If you have received
> > > this communication
> > in
> > > error, please notify the sender and delete all copies of this message.
> > > Persistent Systems Ltd. does not accept any liability for virus
> > > infected mails.
> > >
> >
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is
> the property of Persistent Systems Ltd. It is intended only for the use of
> the individual or entity to which it is addressed. If you are not the
> intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message