Mailing-List: contact dev-help@flume.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flume.apache.org
Date: Tue, 2 Aug 2016 17:47:21 +0000 (UTC)
From: "saravanan sathyamoorthy (JIRA)" <jira@apache.org>
To: dev@flume.apache.org
Message-ID: <JIRA.12895392.1442915780000.202160.1470160041461@Atlassian.JIRA>
In-Reply-To: <JIRA.12895392.1442915780000@Atlassian.JIRA>
References: <JIRA.12895392.1442915780000@Atlassian.JIRA> <JIRA.12895392.1442915780795@arcas>
Subject: [jira] [Commented] (FLUME-2800) Multiline log events for Taildir
 Source
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 02 Aug 2016 17:47:23 -0000


    [ https://issues.apache.org/jira/browse/FLUME-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15404459#comment-15404459 ] 

saravanan sathyamoorthy commented on FLUME-2800:
------------------------------------------------

[~jianjin] were you able to apply the patch?

> Multiline log events for Taildir Source
> ---------------------------------------
>
>                 Key: FLUME-2800
>                 URL: https://issues.apache.org/jira/browse/FLUME-2800
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.6.0, v1.7.0
>            Reporter: David Burgos. Isban (Banco Santander)
>            Priority: Minor
>         Attachments: FLUME-2800-2-Flume1.6.patch, FLUME-2800-2.patch, FLUME-2800-Flume1.6.patch, FLUME-2800.patch
>
>
> This a proposal of implementation to handle multiline log messages for new tailing source FLUME-2498.
> Based on an idea FLUME-2779 MultiLine Deserializer for Spooling DIrectory Source.
> Config.
> * multiLineRegex: Regular expression to handle multiline log messages (grok expressions permitted)
> * grokDictionaryDir: Custom Grok dictionaries directory
> * maxNumberLines: Max number of lines per event in multiline log messages. Default 100. Remaining lines is never transferred to sink. 
> For Regex expressions use Java regex engine.
> Include a functionality for extracting grok expressions into a pure named regex (inspired by the logstash inteceptor)
> By default load the included built-in grok dictionaries with pre-defined patterns.
> https://github.com/aicer/grok
> Attached patch includes a config documentation and unit tests.
> Also attached a completed port/patch for Flume 1.6 a Java 1.6


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)