flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Blade Liu <hafzc...@gmail.com>
Subject Re: Programmatically configuring new source/sink into Flume agent
Date Sun, 28 Sep 2014 08:35:02 GMT
I guess you want to reconfig and run flume agents on-the-fly. Using RPC is
intended to restart agents or handle new config?


2014-09-27 7:58 GMT+08:00 terreyshih <terreyshih@gmail.com>:

> HI, Manohar:
> I don’t understand your examples.  copying/moving file from one directory
> to another is not what source / sinks do.
> thx
> On Sep 26, 2014, at 5:24 AM, Manohar.CS@itcinfotech.com wrote:
> Hi All,
> I have a use-case where in I get source and sink details from user via UI
> page, these config details I want to feed to flum so that programmatically
> it starts processing newly added request for source/sink. When I was
> looking at developer guide
> https://flume.apache.org/FlumeDeveloperGuide.html#rpc-clients-avro-and-thrift
>  there is mention of using RPC client to programmatically get handle to
> flume instance, now I wanted to know how can I programmatically feed these
> user details as and when they arrive ?
> Example – one definition may look like copy files from NFs mount to HDFS
> and one more might look like copy zipped files placed in other directory on
> local system to HDFS sink etc.
> Thanks,
> Manohar.

View raw message