Return-Path: X-Original-To: apmail-flume-user-archive@www.apache.org Delivered-To: apmail-flume-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B80481055F for ; Wed, 29 Jan 2014 04:23:08 +0000 (UTC) Received: (qmail 19112 invoked by uid 500); 29 Jan 2014 04:23:07 -0000 Delivered-To: apmail-flume-user-archive@flume.apache.org Received: (qmail 18612 invoked by uid 500); 29 Jan 2014 04:22:57 -0000 Mailing-List: contact user-help@flume.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flume.apache.org Delivered-To: mailing list user@flume.apache.org Received: (qmail 18603 invoked by uid 99); 29 Jan 2014 04:22:53 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Jan 2014 04:22:53 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of paliwalashish@gmail.com designates 74.125.82.54 as permitted sender) Received: from [74.125.82.54] (HELO mail-wg0-f54.google.com) (74.125.82.54) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Jan 2014 04:22:47 +0000 Received: by mail-wg0-f54.google.com with SMTP id x13so2416797wgg.33 for ; Tue, 28 Jan 2014 20:22:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=3+i/OPR8GTVozFj5b0pimBob2AIRL/WxLBeRiA/6i8s=; b=kg5V7SraenYOFnPT+bg6aZKJrAqsIOSRdThTAxwpdjgAoAs/r7deuqbVbq1n8nz8WU cbefXMNjIci6L0a+g7iQXhpSCTMhniaE2QzeAw0O5fVYWbj3UFXaj197HqV9Z9p8fiXb 5MkUBapw82P4MLYRaoq3ncxYfyQ5SsyV+dAewU5k1M5xsR+dTsLSmeX9KRbOwk02abSq C+XBjnG5qZT1jh+RvlM+Ozkq0bQZ34uBkxjPm5xS3SIprLjQRPTCqP3POpoTryFS+N6T Sj6FEI1HkBbZRwxSpNLrTrDY6bH4kqj8/3vRdzeNIv54rvCDT+dt9VjGgkE3rr85U/Ua tkaQ== MIME-Version: 1.0 X-Received: by 10.194.185.113 with SMTP id fb17mr1044515wjc.29.1390969347627; Tue, 28 Jan 2014 20:22:27 -0800 (PST) Received: by 10.194.46.105 with HTTP; Tue, 28 Jan 2014 20:22:27 -0800 (PST) In-Reply-To: References: Date: Wed, 29 Jan 2014 09:52:27 +0530 Message-ID: Subject: Re: Source Failover and Sink Failover From: Ashish To: user@flume.apache.org Content-Type: multipart/alternative; boundary=047d7bd6bb0edbe01f04f11449a8 X-Virus-Checked: Checked by ClamAV on apache.org --047d7bd6bb0edbe01f04f11449a8 Content-Type: text/plain; charset=ISO-8859-1 On Wed, Jan 29, 2014 at 12:46 AM, Mayur Gupta wrote: > I have two questions related to failover in Flume. > > 1) If the source is not accepting events because of a failure condition, > is there a failover mechanism in Flume other configuring 2 agents to listen > to the events, in which I will get duplicates. > What is the source (not Flume Source) of Events? What happens to the Events if the Flume Source fails? > > 2) I configured a sink processor for failover with 2 sinks, one logger and > another HDFS. If the HDFS sink starts failing (say HDFS is no longer up), > the events are not processed by logger sink. Is this excepted behavior. Is > the failover only works with 2 avro sinks and not in the manner I have > configured? > IMHO, this should not be the case. Anything in the logs? > > I will appreciate it if somebody can point me in the right direction. > > -Mayur > -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal --047d7bd6bb0edbe01f04f11449a8 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable



On Wed, Jan 29, 2014 at 12:46 AM, Mayur Gupta <= ;mayur.gupta81= @gmail.com> wrote:
I have two questions related to failover i= n Flume.

1) If the source is not accepting events because of a failure condition, is= there a failover mechanism in Flume other configuring 2 agents to listen t= o the events, in which I will get duplicates.

What is the source (not Flume Source) of Events? What h= appens to the Events if the Flume Source fails?=A0
=A0

2) I c= onfigured a sink processor for failover with 2 sinks, one logger and anothe= r HDFS. If the HDFS sink starts failing (say HDFS is no longer up), the eve= nts are not processed by logger sink. Is this excepted behavior. Is the fai= lover only works with 2 avro sinks and not in the manner I have configured?=

IMHO, this should not be the case. A= nything in the logs?
=A0

I will appreciate it i= f somebody can point me in the right direction.

-Mayur=



--
thanks
as= hish

Blog: http://www.= ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
--047d7bd6bb0edbe01f04f11449a8--