flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Baris Akgun (Garanti Teknoloji)" <BarisA...@garanti.com.tr>
Subject RE: Flume High Available Methods
Date Thu, 31 Mar 2016 07:59:05 GMT
Thanks a lot.

Yes, we will plan to use kafka channel as you said. As I understand from your comments, our
current structure seems good for HA.

Regards,
Barış



From: Gonzalo Herreros [mailto:gherreros@gmail.com]
Sent: Thursday, March 31, 2016 10:44 AM
To: user
Subject: Re: Flume High Available Methods

For me the best practice is what the big vendors do and recommend.
Your solution is of deploying multiple identical agents sharing the group in the source is
fine as long as you have a durable channel and are ok with some messages getting delayed (or
even lost) when a node goes down.

If you want full failover then you need to use the KafkaChannel, that way Flume is backed
by Kafka and any agent can sink an event sources by any other agent.

Regards,
Gonzalo


On 30 March 2016 at 11:53, Baris Akgun (Garanti Teknoloji) <BarisAkgu@garanti.com.tr<mailto:BarisAkgu@garanti.com.tr>>
wrote:
Hi ,

I am trying to create flume agent in high availability architecture with using kafka source
and setting group id configuration parameter in my flume agent conf file.

What is the best practise for doing high availability architecture in flume? Is there anybody
who created flume agent in HA structure ?

Thanks.

Barış.


Barış Akgün
Analitik Veri Ambarı ve Büyük Veri Yönetimi
Uzman

Tel

:

Dahili

:

Faks

:



Bu mesaj ve ekleri, mesajda gonderildigi belirtilen kisi/kisilere ozeldir ve gizlidir. Bu
mesajin muhatabi olmamaniza ragmen tarafiniza ulasmis olmasi halinde mesaj iceriginin gizliligi
ve bu gizlilik yukumlulugune uyulmasi zorunlulugu tarafiniz icin de soz konusudur. Mesaj ve
eklerinde yer alan bilgilerin dogrulugu ve guncelligi konusunda gonderenin ya da sirketimizin
herhangi bir sorumlulugu bulunmamaktadir. Sirketimiz mesajin ve bilgilerinin size degisiklige
ugrayarak veya gec ulasmasindan, butunlugunun ve gizliliginin korunamamasindan, virus icermesinden
ve bilgisayar sisteminize verebilecegi herhangi bir zarardan sorumlu tutulamaz.

This message and attachments are confidential and intended solely for the individual(s) stated
in this message. If you received this message although you are not the addressee, you are
responsible to keep the message confidential. The sender has no responsibility for the accuracy
or correctness of the information in the message and its attachments. Our company shall have
no liability for any changes or late receiving, loss of integrity and confidentiality, viruses
and any damages caused in anyway to your computer system.

Bu mesaj ve ekleri, mesajda gonderildigi belirtilen kisi/kisilere ozeldir ve gizlidir. Bu
mesajin muhatabi olmamaniza ragmen tarafiniza ulasmis olmasi halinde mesaj iceriginin gizliligi
ve bu gizlilik yukumlulugune uyulmasi zorunlulugu tarafiniz icin de soz konusudur. Mesaj ve
eklerinde yer alan bilgilerin dogrulugu ve guncelligi konusunda gonderenin ya da sirketimizin
herhangi bir sorumlulugu bulunmamaktadir. Sirketimiz mesajin ve bilgilerinin size degisiklige
ugrayarak veya gec ulasmasindan, butunlugunun ve gizliliginin korunamamasindan, virus icermesinden
ve bilgisayar sisteminize verebilecegi herhangi bir zarardan sorumlu tutulamaz.

This message and attachments are confidential and intended solely for the individual(s) stated
in this message. If you received this message although you are not the addressee, you are
responsible to keep the message confidential. The sender has no responsibility for the accuracy
or correctness of the information in the message and its attachments. Our company shall have
no liability for any changes or late receiving, loss of integrity and confidentiality, viruses
and any damages caused in anyway to your computer system.
Mime
View raw message