nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <>
Subject Re: howto dynamically change the PutHDFS target directory
Date Mon, 18 Apr 2016 14:02:45 GMT

If I am understanding correctly I think this can be done today... The
Directory property on PutHDFS supports expression language, so you could
set it to a value like:


This could be set directly in PutHDFS, although it is also a common pattern
to stick an UpdateAttribute processor in front of PutHDFS and set filename
and hadoop.dir attributes, and then in PutHDFS reference those as
${filename} and ${hadoop.dir}

The advantage to the UpdateAttribute approach is that you can have a single
PutHDFS processor that actually writes to many different locations.

Hope that helps.


On Mon, Apr 18, 2016 at 2:53 PM, Oleg Zhurakousky <> wrote:

> Mike
> Indeed a very common requirement and we should support it.
> Would you mind raising a JIRA for it?
> Cheers
> Oleg
> On Apr 18, 2016, at 9:50 AM, Mike Harding <> wrote:
> Hi All,
> I have a requirement to write a data stream into HDFS, where the flowfiles
> received per day are group into a directory. e.g. so I would end up with a
> folder structure as follows:
> data/18-04-16
> data/19-04-16
> data/20-04-16 ... etc
> Currently I can specify in the config for the putHDFS processor a target
> directory but I want this to change and point to a new directory as each
> day ends.
> So using nifi id like to 1) be able to create new directories in HDFS
> (although I could potentially write a bash script to do the directory
> creation) and 2) change the target directory as the day changes.
> Any help much appreciated,
> Mike

View raw message