flume-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhenhuanli708 (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLUME-3235) Flume TAILDIR source lost last line when file upload from windows
Date Thu, 03 May 2018 09:07:00 GMT

     [ https://issues.apache.org/jira/browse/FLUME-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

zhenhuanli708 updated FLUME-3235:
---------------------------------
    Labels:   (was: pull-request-available)

> Flume TAILDIR source lost last line when file upload from windows
> -----------------------------------------------------------------
>
>                 Key: FLUME-3235
>                 URL: https://issues.apache.org/jira/browse/FLUME-3235
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: 1.8.0
>         Environment: flume ng 1.8
> centos 7.3
>            Reporter: zhenhuanli708
>            Priority: Major
>         Attachments: flume.PNG, flume_result.PNG
>
>   Original Estimate: 96h
>  Remaining Estimate: 96h
>
> 1.upload windows file to /root/windows_file.txt.
> file conent is 
> !flume.PNG!
> The last line "end_line" end without \n
> 2. config flume ng 
> {quote}flow.channels = c1
> flow.sources = r1
> flow.sinks = k1
> flow.channels.c1.type = memory
> flow.sources.r1.type = TAILDIR
> flow.sources.r1.filegroups = f1
> flow.sources.r1.filegroups.f1 = /root/windows_file.txt
> flow.sources.r1.batchSize = 100
> flow.sinks.k1.type = logger
> flow.sources.r1.channels = c1
> flow.sinks.k1.channel = c1
> {quote}
>  
> 3. remove /root/.flume/taildir_position.json then run this config flume ng:
> {quote}nohup bin/flume-ng agent --conf conf/ -f conf/flume-conf_windows_end_file.properties
-n flow -Dflume.root.logger=DEBUG,console &
> {quote}
> 4.check nohup.out : this file does not contains end_line.
>    !flume_result.PNG!
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@flume.apache.org
For additional commands, e-mail: issues-help@flume.apache.org


Mime
View raw message