flume-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhenhuanli708 (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLUME-3235) Flume TAILDIR source lost last line when file upload from windows
Date Thu, 03 May 2018 09:02:00 GMT
zhenhuanli708 created FLUME-3235:
------------------------------------

             Summary: Flume TAILDIR source lost last line when file upload from windows
                 Key: FLUME-3235
                 URL: https://issues.apache.org/jira/browse/FLUME-3235
             Project: Flume
          Issue Type: Bug
          Components: Sinks+Sources
    Affects Versions: 1.8.0
         Environment: flume ng 1.8

centos 7.3
            Reporter: zhenhuanli708
         Attachments: flume.PNG, flume_result.PNG

1.upload windows file to /root/windows_file.txt.

file conent is 

!flume.PNG!

The last line "end_line" end without \n

2. config flume ng 
{quote}flow.channels = c1
flow.sources = r1
flow.sinks = k1

flow.channels.c1.type = memory

flow.sources.r1.type = TAILDIR
flow.sources.r1.filegroups = f1
flow.sources.r1.filegroups.f1 = /root/windows_file.txt
flow.sources.r1.batchSize = 100

flow.sinks.k1.type = logger

flow.sources.r1.channels = c1
flow.sinks.k1.channel = c1
{quote}
 

3. remove /root/.flume/taildir_position.json then run this config flume ng:
{quote}nohup bin/flume-ng agent --conf conf/ -f conf/flume-conf_windows_end_file.properties
-n flow -Dflume.root.logger=DEBUG,console &
{quote}
4.check nohup.out : this file does not contains end_line.

   !flume_result.PNG!

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@flume.apache.org
For additional commands, e-mail: issues-help@flume.apache.org


Mime
View raw message