nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Takanobu Asanuma <tasan...@yahoo-corp.jp>
Subject RE: How to ingest files into HDFS via Apache NiFi from non-hadoop environment
Date Wed, 28 Jun 2017 04:13:46 GMT
Hello Mothi86,

I think you can achieve it by using HttpFS on HDFS side. It is a part of hadoop library and
a proxy server for HDFS.
https://hadoop.apache.org/docs/stable/hadoop-hdfs-httpfs/index.html

In your case, running HttpFS server on the management nodes or the edge node would be good.
And set 'webhdfs://{HttpFS hostname}:{port}' to 'fs.defaultFS' in core-site.xml for NiFi's
HDFS processors. Then, your NiFi cluster only need to access the HttpFS server and can access
HDFS from non-hadoop environment.

Regards,
Takanobu
Mime
View raw message