nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jamesgreen <james.gr...@standardbank.co.za>
Subject Compression of Data in HDFS
Date Wed, 06 Apr 2016 11:12:51 GMT
I am trying to compress a whole lot of files from my HDFS and write to
another folder on the HDFS
My Folder Structure is as follows:
\landing\databasename\prodeiw_arc\tablename\_SUCCESS
\landing\databasename\prodeiw_arc\tablename\part-m-00000

\landing\databasename\prodeiw_arc\tablename2\_SUCCESS
\landing\databasename\prodeiw_arc\tablename2\part-m-00000

I am trying to compress to the following
\landing\compressed\prodeiw_arc\tablename\_SUCCESS
\landing\compressed\prodeiw_arc\tablename\part-m-00000

\landing\compressed\prodeiw_arc\tablename2\_SUCCESS
\landing\compressed\prodeiw_arc\tablename2\part-m-00000

I have found that it compresses to 
\landing\compressed\prodeiw_arc\_SUCCESS
\landing\compressed\prodeiw_arc\tablename\part-m-00000

it will then continue to overwrite. Is there anyway I can keep the directory
structure when doing a PutHDFS?

Thanks and Regards




--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Compression-of-Data-in-HDFS-tp8821.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Mime
View raw message