flume-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "tinawenqiao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLUME-2961) Make TaildirSource work with multiline
Date Tue, 26 Jul 2016 08:39:20 GMT

     [ https://issues.apache.org/jira/browse/FLUME-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

tinawenqiao updated FLUME-2961:
-------------------------------
    Description: 
TaidirSource defaults to LINE, this has issue when multiline log events like stack traces
and have request/responses. Following part is Java traceback logs. We expect to have log line
start regex Key to aggregate all the log lines till the next regex key is found.
2016-07-16 14:59:43,956 ERROR lifecycleSupervisor-1-7 LifecycleSupervisor.run - Unable to
start EventDrivenSourceRunner: { source:cn.yottabyte.flume.source.http.HTTPSource{name:sourceHttp,state:IDLE}
} - Exception follows.
java.lang.IllegalStateException: Running HTTP Server found in source: sourceHttp before I
started one. Will not attempt to start.
    at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
    at cn.yottabyte.flume.source.http.HTTPSource.startHttpSourceServer(HTTPSource.java:170)
    at cn.yottabyte.flume.source.http.HTTPSource.start(HTTPSource.java:166)
    at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
    at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

  was:
1. TaidirSource defaults to LINE, this has issue when multiline log events like stack traces
and have request/responses.
2. Proposal to have log line start regex Key to aggregate all the log lines till the next
regex key is found.


> Make TaildirSource work with multiline
> --------------------------------------
>
>                 Key: FLUME-2961
>                 URL: https://issues.apache.org/jira/browse/FLUME-2961
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.7.0
>            Reporter: tinawenqiao
>            Assignee: tinawenqiao
>             Fix For: v1.7.0
>
>         Attachments: FLUME-2961_1.patch
>
>
> TaidirSource defaults to LINE, this has issue when multiline log events like stack traces
and have request/responses. Following part is Java traceback logs. We expect to have log line
start regex Key to aggregate all the log lines till the next regex key is found.
> 2016-07-16 14:59:43,956 ERROR lifecycleSupervisor-1-7 LifecycleSupervisor.run - Unable
to start EventDrivenSourceRunner: { source:cn.yottabyte.flume.source.http.HTTPSource{name:sourceHttp,state:IDLE}
} - Exception follows.
> java.lang.IllegalStateException: Running HTTP Server found in source: sourceHttp before
I started one. Will not attempt to start.
>     at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
>     at cn.yottabyte.flume.source.http.HTTPSource.startHttpSourceServer(HTTPSource.java:170)
>     at cn.yottabyte.flume.source.http.HTTPSource.start(HTTPSource.java:166)
>     at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
>     at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>     at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>     at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message