flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Inder Pall <inder.p...@gmail.com>
Subject Re: Source and Sink running on different machine
Date Fri, 29 Jun 2012 10:34:19 GMT
what's the sink you have given in the first agent that you have setup.
To connect the pipelines AFAIK you need to have a
<Somesource><channel><AvroSink> on the first agent and then
<AvroSource><channel><Whatever sink you need> at the second agent.

If you have configured as mentioned below then it's not what you are asking
->

If i get your question right you want source and Sink for a single agent to
be able to run on different machines and share the same common distributed
channel. Though i havent tested it but JDBC channel might address this
usecase.

Btw, it would be nice to understand the motivation behind such a
configuration

Thanks,
- inder

On Fri, Jun 29, 2012 at 3:53 PM, shekhar sharma <shekhar2581@gmail.com>wrote:

> Hi Mohammad,
> Thanks for quick reply.
> I think its possible.
> What i did, in one machine i have created a properties file which consist
> of only avro source properties (like host name and port and channel)
>
> then in another machine i have created another properties file which
> consist of avro sink properties( host name and port to connect)..
> In the log file where avro source is running, you can see that the
> connection has been established between the two.
>
> Regards,
> Som Shekhar
>
>
>
> On Fri, Jun 29, 2012 at 3:34 PM, Mohammad Tariq <dontariq@gmail.com>wrote:
>
>> Hello Shekhar,
>>
>>          I don't think it is possible in Flume-NG. An agent is
>> composed of source, channel and sink and together they make a flow.
>> Moreover, each agent is an atomic entity in a Flume-NG flow. But I
>> need a green signal from the experts on this.
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>> On Fri, Jun 29, 2012 at 3:11 PM, shekhar sharma <shekhar2581@gmail.com>
>> wrote:
>> > Hello ,
>> > I am using flume-ng(1.2.0)
>> >
>> >
>> > Is it possible to have source and sink running on different machine?
>> > For example:
>> > In my flume agent, if i am using exec source and logger sink, is it
>> possible
>> > to have exec source running on machine 1 and logger sink running on
>> machine
>> > 2?
>> >
>> > Or is it always necessary to have the source and sink tied up together
>> on a
>> > single machine?
>> >
>> >
>> >
>> > Regards,
>> > Som  Shekhar
>>
>
>


-- 
Thanks,
- Inder
  Tech Platforms @Inmobi
  Linkedin - http://goo.gl/eR4Ub

Mime
View raw message