flume-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bessenyei Balázs Donát (JIRA) <j...@apache.org>
Subject [jira] [Updated] (FLUME-2653) Allow inUseSuffix to be null/empty
Date Wed, 20 Jul 2016 13:51:20 GMT

     [ https://issues.apache.org/jira/browse/FLUME-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Bessenyei Balázs Donát updated FLUME-2653:
    Labels: docs-missing hdfssink  (was: hdfssink)

> Allow inUseSuffix to be null/empty
> ----------------------------------
>                 Key: FLUME-2653
>                 URL: https://issues.apache.org/jira/browse/FLUME-2653
>             Project: Flume
>          Issue Type: Improvement
>          Components: Sinks+Sources
>    Affects Versions: v1.5.1
>            Reporter: Andrew Jones
>            Assignee: bimal tandel
>              Labels: docs-missing, hdfssink
>             Fix For: v1.7.0
>         Attachments: FLUME-2653.patch
> At the moment, it doesn't seem possible to set the null/empty. We've tried {{''}} which
just adds the quotes to the end, and setting to nothing, which just uses the default {{.tmp}}.
> We want the _in use_ file to have the same name as the _closed_ file, so we can read
from files that are in use without the file moving from underneath us. In our use case, we
know that an in use file is still readable and parseable, because it is just text with a JSON
document per line.
> It looks like [the HDFS sink code|https://github.com/apache/flume/blob/542b1695033d330eb00ae81713fdc838b88332b6/flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java#L618]
can handle this change already, but at the moment there is no way to set the {{bucketPath}}
and {{targetPath}} to be the same.

This message was sent by Atlassian JIRA

View raw message