Return-Path: X-Original-To: apmail-incubator-flume-user-archive@minotaur.apache.org Delivered-To: apmail-incubator-flume-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9BA319B16 for ; Mon, 14 Nov 2011 12:13:19 +0000 (UTC) Received: (qmail 31685 invoked by uid 500); 14 Nov 2011 12:13:19 -0000 Delivered-To: apmail-incubator-flume-user-archive@incubator.apache.org Received: (qmail 31667 invoked by uid 500); 14 Nov 2011 12:13:19 -0000 Mailing-List: contact flume-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: flume-user@incubator.apache.org Delivered-To: mailing list flume-user@incubator.apache.org Received: (qmail 31659 invoked by uid 99); 14 Nov 2011 12:13:19 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 14 Nov 2011 12:13:19 +0000 X-ASF-Spam-Status: No, hits=0.7 required=5.0 tests=RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [217.70.183.196] (HELO relay4-d.mail.gandi.net) (217.70.183.196) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 14 Nov 2011 12:13:09 +0000 X-Originating-IP: 217.70.178.142 Received: from mfilter14-d.gandi.net (mfilter14-d.gandi.net [217.70.178.142]) by relay4-d.mail.gandi.net (Postfix) with ESMTP id BCBBB1720F0 for ; Mon, 14 Nov 2011 13:12:48 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at mfilter14-d.gandi.net Received: from relay4-d.mail.gandi.net ([217.70.183.196]) by mfilter14-d.gandi.net (mfilter14-d.gandi.net [10.0.15.180]) (amavisd-new, port 10024) with ESMTP id NgYFvEZmGr8S for ; Mon, 14 Nov 2011 13:12:47 +0100 (CET) X-Originating-IP: 209.85.210.175 Received: from mail-iy0-f175.google.com (mail-iy0-f175.google.com [209.85.210.175]) (Authenticated sender: gmail@vafer.org) by relay4-d.mail.gandi.net (Postfix) with ESMTPSA id 04FF11721B9 for ; Mon, 14 Nov 2011 13:12:46 +0100 (CET) Received: by iahk25 with SMTP id k25so5931820iah.6 for ; Mon, 14 Nov 2011 04:12:45 -0800 (PST) MIME-Version: 1.0 Received: by 10.42.151.68 with SMTP id d4mr22176694icw.36.1321272765167; Mon, 14 Nov 2011 04:12:45 -0800 (PST) Received: by 10.42.170.1 with HTTP; Mon, 14 Nov 2011 04:12:45 -0800 (PST) In-Reply-To: References: Date: Mon, 14 Nov 2011 13:12:45 +0100 Message-ID: Subject: Re: How to tail a flume stream? From: Torsten Curdt To: flume-user@incubator.apache.org Content-Type: text/plain; charset=UTF-8 X-Virus-Checked: Checked by ClamAV on apache.org > As far as I understood a flume agent can sent to multiple sinks. Can't you > just configure two sinks, one as HDFS and on as normal file system? I'm > interested in that functionality as well. Does anybody know? That's creates a copy but you want a buffer that gets flushed once HDFS is available again. Plus you don't want duplicates. cheers, Torsten