nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Bende <bbe...@gmail.com>
Subject Re: put pictures from remote server into hdfs
Date Thu, 22 Mar 2018 16:41:34 GMT
Hello,

It would probably be best to use GetSFTP -> PutHDFS.

No need to write the files out to local disk somewhere else with
PutFile, they can go straight to HDFS.

The filename in HDFS will be the "filename" attribute of the flow
file, which GetSFTP should be setting to the filename it picked up.

If you need a different filename, you can stick an UpdateAttribute
before PutHDFS and change the filename attribute to whatever makes
sense.

-Bryan


On Thu, Mar 22, 2018 at 12:18 PM, 李 磊 <lei.li@ruisdata.com> wrote:
> Hi all,
>
>
>
> It is my requirement that put pictures from remote server(not in nifi
> cluster) into hdfs.
>
> First I use the GetSFTP and PutFile to get pictures to local, and then use
> ExecuteFlumeSource and ExecuteFlumeSink to put pictures into hdfs from
> local.
>
>
>
> However, there is a problem that the name of pictures that put into hdfs
> cannot keep the same with local.
>
>
>
> Could you tell me the way to keep the name same or a better way to put
> pictures into hdfs from remote server with nifi?
>
>
>
> Thanks!

Mime
View raw message