Return-Path: X-Original-To: apmail-flume-user-archive@www.apache.org Delivered-To: apmail-flume-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B56F2D7A3 for ; Wed, 5 Sep 2012 19:15:37 +0000 (UTC) Received: (qmail 47386 invoked by uid 500); 5 Sep 2012 19:15:37 -0000 Delivered-To: apmail-flume-user-archive@flume.apache.org Received: (qmail 47328 invoked by uid 500); 5 Sep 2012 19:15:37 -0000 Mailing-List: contact user-help@flume.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flume.apache.org Delivered-To: mailing list user@flume.apache.org Received: (qmail 47320 invoked by uid 99); 5 Sep 2012 19:15:37 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Sep 2012 19:15:37 +0000 X-ASF-Spam-Status: No, hits=-0.1 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of David.Cochran@bsee.gov designates 137.227.82.13 as permitted sender) Received: from [137.227.82.13] (HELO smtp3.doi.gov) (137.227.82.13) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 05 Sep 2012 19:15:29 +0000 Received: from smtp3.smgd.doi.gov (10.10.82.13) by smtp3.doi.gov (10.10.82.23) with Microsoft SMTP Server id 14.2.298.4; Wed, 5 Sep 2012 13:15:08 -0600 X-AuditID: 0a0a520d-b7f0f6d000001f50-60-50479a0cfc78 Received: from mx01.mms.gov (mx01.mms.gov [192.135.178.28]) by smtp3.smgd.doi.gov (Hello) with SMTP id 54.11.08016.C0A97405; Wed, 5 Sep 2012 12:29:32 -0600 (MDT) Received: from unknown (HELO IMSHEXCON01.service.agency.mms.pri) ([10.22.54.20]) by mx01.mms.gov with ESMTP; 05 Sep 2012 12:29:32 -0600 Received: from imsdexpri10.service.agency.mms.pri ([10.17.54.103]) by IMSHEXCON01.service.agency.mms.pri with Microsoft SMTPSVC(6.0.3790.4675); Wed, 5 Sep 2012 14:29:32 -0400 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-Class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----_=_NextPart_001_01CD8B94.63F3ED3B" Subject: RE: conf help needed Date: Wed, 5 Sep 2012 12:29:31 -0600 Message-ID: In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: conf help needed Thread-Index: Ac2BTMzeGXv4nBGuTsuZfZg4YXrzhAKRme/g References: From: "Cochran, David M (Contractor)" To: X-OriginalArrivalTime: 05 Sep 2012 18:29:32.0231 (UTC) FILETIME=[6491CD70:01CD8B94] X-CFilter-Loop: Reflected X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrNIsWRmVeSWpSXmKPExsXCxfTrlS7PLPcAg8MNJhbrz8xidmD0+PV7 EUsAYxSXTUpqTmZZapG+XQJXxoy/h1kLdttW9N84wtzAONesi5GTQ0LARGLt4pvMELaYxIV7 69m6GLk4hASWMEocuP6WFcLpYpRYte4rC4SzkVHi0/eDjBAtehLLFk9hAbGZBbQkjlxqAovz CghKnJz5BCoeLtFz6w4TiC0sICvx7WkDmM0ioCLx69YfqPpwif235rOC2JwCgRK/9x5ig5gv KLFo9h648/7teggU5+AQEZCT2P47BcI0krh5Duq0B4wSiy7cBRvPJmAnMa9tG9gYEQFpie1f roKtEhJQlnjb8o4VYmSwxNydt5kgbEmJgytusExgFJ+F5JtZSL6ZheSbWUCrmYG+b9vICBHW lli28DUzhK0r8f/5HGZk8QWM7KsYhYpzSwqM9Ypz01P0UvIz9dLzyzYxgqKQK4h3B+POPZaH GAU4GJV4eOv73AOEWBPLiitzDzFKcDArifCu7gYK8aYkVlalFuXHF5XmpBYfYpTmYFES57UG qRZITyxJzU5NLUgtgskycXBKNTA2aT+IuXRN0m5J4JSOzJfXL5XUTpgdq1O1L3NVusCel6zt p3fdCn4VfPrz/z0NtUahet8WlVxM3NK+VaTIQWOWT5C7f38L1z2H/kkuz1pFK3O2T3NKq7Ne d+jIowMCk80vnOg4ksTBv1FtaZXXVx2tWqE1PzhqIxp/Rh4Q40qIdmeeF7ZCo0qJpTgj0VCL uag4EQC2YYthvgIAAA== X-Virus-Checked: Checked by ClamAV on apache.org ------_=_NextPart_001_01CD8B94.63F3ED3B Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Sorry for the long delay... Isaac threw a wrench into things. =20 Thanks for the replies... Okay since what I'm looking for isn't implemented yet, I can manually set the directories for each host without a big deal... =20 =20 How about the output filenames? Can they be manually set somehow, perhaps just set a prefix and then let flume append it with the numbered structure? =20 =20 ie.. app1_log_xxxxxxxxxx app2_log_xxxxxxxxxx app2_out_xxxxxxxxxx =20 or would I need to create a directory for each and deal with the 672153171-1 filenames? =20 Thanks,=20 Dave =20 =20 From: Will McQueen [mailto:will@cloudera.com]=20 Sent: Thursday, August 23, 2012 11:31 AM To: user@flume.apache.org Subject: Re: conf help needed =20 Hi David, There's currently an open, unassigned ticket for what you're asking: FLUME-1295: RollingFileSink needs to be able to construct directory path based on escape sequence Cheers, Will On Thu, Aug 23, 2012 at 9:17 AM, Bhaskar V. Karambelkar wrote: Looking at the flume documentation, looks like dynamic paths are only supported for the HDFS sink and not the file_roll sink. You can use a Multiplexing channel selector to achieve the same effect for file_roll sink, but the possible hosts has to be a predetermined list and not dynamic. =20 ------_=_NextPart_001_01CD8B94.63F3ED3B Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Sorry for the long delay… Isaac threw a wrench into = things.

 

Thanks for the replies…  Okay since what I’m looking = for isn’t implemented yet, I can manually set the directories for = each host without a big deal…  

 

How about the output filenames?  Can they be manually set = somehow, perhaps just set a prefix and then let flume append it with the = numbered structure? 

 

ie..  app1_log_xxxxxxxxxx

=         app2_log_xxxxxxxxxx<= /o:p>

         = app2_out_xxxxxxxxxx

 

or would I need to create a directory for each and deal with the = 672153171-1 filenames?

 

Thanks,

Dave

 

 

From:= = Will McQueen [mailto:will@cloudera.com]
Sent: Thursday, = August 23, 2012 11:31 AM
To: = user@flume.apache.org
Subject: Re: conf help = needed

 

Hi = David,

There's currently an open, unassigned ticket for what = you're asking:
     FLUME-1295: RollingFileSink = needs to be able to construct directory path based on escape = sequence

Cheers,
Will

On Thu, Aug 23, 2012 at 9:17 AM, Bhaskar V. = Karambelkar <bhaskarvk@gmail.com> wrote:

Looking at the flume documentation, looks like dynamic = paths are only supported for the HDFS sink and not the file_roll = sink.

You can use a Multiplexing = channel selector to achieve the same effect for file_roll sink, but the = possible hosts has to be a

predetermined list and not = dynamic.

 

------_=_NextPart_001_01CD8B94.63F3ED3B--