flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Durgapal <durgapalmo...@gmail.com>
Subject Re: When to stop an agent with a Spool Directory source
Date Fri, 23 Jan 2015 06:10:50 GMT
correction:

 instead of "while starting the sink" i meant "while starting the agent".



On Fri, Jan 23, 2015 at 11:39 AM, Mohit Durgapal <durgapalmohit@gmail.com>
wrote:

>
> Currently I am using cloudera to start/stop flume agents.
>
> But earlier I used to provide the source& sink configuration while
> starting the sink from terminal with a command like below:
>
> bin/flume-ng agent --conf conf --conf-file example.conf --name a1 -Dflume.root.logger=INFO,console
>
> Regards
> Mohit
>
> On Fri, Jan 23, 2015 at 11:01 AM, Carlotta Hicks <Carlotta.Hicks@sas.com>
> wrote:
>
>>  How/where are you starting your sources and sinks?
>>
>>
>>
>> *From:* Mohit Durgapal [mailto:durgapalmohit@gmail.com]
>> *Sent:* Friday, January 23, 2015 12:20 AM
>> *To:* user
>> *Subject:* Re: When to stop an agent with a Spool Directory source
>>
>>
>>
>> I am not sure about your use case but I am writing about my experiences
>> with flume.
>> Generally you would use flume for streaming data and will not stop the
>> components ever. You have  source & sinks running all the time so that as
>> soon as an application writes into the spool directory, the source reads
>> it(as per the flume agent configuration) and puts it the in the channel
>> from where sink consumes it.
>>
>> Otherwise, I can't think of anything other than a shell script to run
>> periodically to start-stop the agent.
>>
>>  Regards
>>
>> Mohit
>>
>>
>>
>> On Fri, Jan 23, 2015 at 10:28 AM, Carlotta Hicks <Carlotta.Hicks@sas.com>
>> wrote:
>>
>>
>>
>> I am a new Flume user.  I have a java application where  I would like to
>> start a spool directory source, file channel and hbase sink.
>>
>>
>>
>> After I start the components and copy a file into the spooldir directory,
>> programmatically, how do I know when to
>>
>> stop the components?
>>
>>
>>
>>
>>
>
>

Mime
View raw message