Mailing-List: contact users-help@nifi.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@nifi.apache.org
MIME-Version: 1.0
In-Reply-To: <CALxHsARop97uvWf9iO-wnuQmr0GdGp0te5YfGVPdO96NuFV5hA@mail.gmail.com>
References: <CALxHsAQ-QtO-AeFu+KXfvLyy9oLok_ggcrYs26vncDXKi2Vf9A@mail.gmail.com>
 <CADT6bVuf2b0o7X1pnf-_0b-6+sQU37+gopiQDwJ5Uqcp28TWoQ@mail.gmail.com> <CALxHsARop97uvWf9iO-wnuQmr0GdGp0te5YfGVPdO96NuFV5hA@mail.gmail.com>
From: Joe Witt <joe.witt@gmail.com>
Date: Tue, 26 Sep 2017 21:05:16 -0400
Message-ID: <CALJK9a6ecsBnymb7Smhk1xbi9N38QOBVw6=uMZ=xNdHMiELSAg@mail.gmail.com>
Subject: Re: Using MiNiFi as a library from with a Java program
To: users@nifi.apache.org
Content-Type: text/plain; charset="UTF-8"
archived-at: Wed, 27 Sep 2017 01:05:24 -0000

Praveen

I think the direction of MiNiFi and where it can head will support
this case nicely.  Today though I dont think we offer the simple
library model you're looking for.  Fortunately, for Apache Kafka
specifically their community has developed Kafka Connect [1] which
sounds like it could be just what you need.

Take a look at that and if that gets you where you want to be then
great.  If not, or I've misunderstood the ask please let us know.

[1] https://kafka.apache.org/documentation/#connect

Thanks
Joe

On Tue, Sep 26, 2017 at 8:50 PM, p pathiyil <pathiyil@gmail.com> wrote:
> Hi Aldrin,
>
> I am looking at a couple of different flows, but to take the most simple
> scenario, I would like to leverage the PublishToKafka_0_10 processor of
> MiNiFi alone. The application will obtain some data, send it to a Processor
> that has a simple upstream connection interface like a socket or in memory
> buffer (so that conversion to FlowFiles can be taken care of in that
> Processor) and then use PublishToKafka_0_10 to directly publish to a Kafka
> cluster. Will that be easy enough to do ?
>
> Thanks,
> Praveen.
>
>
>
>
> On Wed, Sep 27, 2017 at 12:24 AM, Aldrin Piri <aldrinpiri@gmail.com> wrote:
>>
>> Hello,
>>
>> Neither NiFi nor MiNiFi (Java) is currently packaged as such.  Would you
>> be able to expand upon your use case of publishing to Kafka and/or what
>> facets of MiNiFi you are looking to utilize?  There may be other avenues
>> that get you toward your solution.
>>
>> Thanks,
>> Aldrin
>>
>> On Tue, Sep 26, 2017 at 1:03 PM, p pathiyil <pathiyil@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> I am starting to look at MiNiFi for a use case that involves publishing
>>> to Kafka. Is it feasible to use MiNiFi as a library (or set of libraries)
>>> from within another Java program ? The primary reason for looking at such an
>>> approach is the desire to hide the MiNiFi installation / process management
>>> from the users of my application.
>>>
>>> Thanks.
>>>
>>
>