nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Parul Agrawal <parulagrawa...@gmail.com>
Subject Re: Need help in nifi- flume processor
Date Thu, 08 Oct 2015 06:30:33 GMT
Hello Bryan,

Thank you very much for your response.

Is it possible to have customized flume source and sink in Nifi?
I have my own customized source and sink? I followed below steps to add my
own customized source but it did not work.

1) Created Maven project and added customized source. (flume.jar was
created after this step)
2) Added flume.jar file to nifi-0.3.0/lib folder.
3) Added flume source processor with the below configuration

Property           Value
Source Type         com.flume.source.Source
Agent Name      a1
Source Name         k1.

But I am getting the below error in Flume Source Processor.
"Failed to run validation due to java.lang.NoClassDefFoundError :
/org/apache/flume/PollableSource."

Can you please help me in this regard. Any step/configuration I missed.

Thanks and Regards,
Parul


On Wed, Oct 7, 2015 at 6:57 PM, Bryan Bende <bbende@gmail.com> wrote:

> Hello,
>
> The NiFi Flume processors are for running Flume sources and sinks with in
> NiFi. They don't communicate with an external Flume process.
>
> In your example you would need an ExecuteFlumeSource configured to run the
> netcat source, connected to a ExecuteFlumeSink configured with the logger.
>
> -Bryan
>
> On Wednesday, October 7, 2015, Parul Agrawal <parulagrawal14@gmail.com>
> wrote:
>
>> Hi,
>>
>> I was trying to run Nifi Flume processor with the below mentioned
>> details but not could bring it up.
>>
>> I already started flume with the sample configuration file
>> =============================================
>> # example.conf: A single-node Flume configuration
>>
>> # Name the components on this agent
>> a1.sources = r1
>> a1.sinks = k1
>> a1.channels = c1
>>
>> # Describe/configure the source
>> a1.sources.r1.type = netcat
>> a1.sources.r1.bind = localhost
>> a1.sources.r1.port = 44444
>>
>> # Describe the sink
>> a1.sinks.k1.type = logger
>>
>> # Use a channel which buffers events in memory
>> a1.channels.c1.type = memory
>> a1.channels.c1.capacity = 1000
>> a1.channels.c1.transactionCapacity = 100
>>
>> # Bind the source and sink to the channel
>> a1.sources.r1.channels = c1
>> a1.sinks.k1.channel = c1
>> =============================================
>>
>> Command used to start flume : $ bin/flume-ng agent --conf conf
>> --conf-file example.conf --name a1 -Dflume.root.logger=INFO,console
>>
>> In the Nifi browser of ExecuteFlumeSink following configuration was done:
>> Property           Value
>> Sink Type         logger
>> Agent Name      a1
>> Sink Name         k1.
>>
>> Event is sent to the flume using:
>> $ telnet localhost 44444
>> Trying 127.0.0.1...
>> Connected to localhost.localdomain (127.0.0.1).
>> Escape character is '^]'.
>> Hello world! <ENTER>
>> OK
>>
>> But I could not get any data in the nifi flume processor. Request your
>> help in this.
>> Do i need to change the example.conf file of flume so that Nifi Flume
>> Sink should get the data.
>>
>> Thanks and Regards,
>> Parul
>>
>
>
> --
> Sent from Gmail Mobile
>

Mime
View raw message