nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tripathi, Shiv Deepak" <shiv.deepak.tripa...@philips.com>
Subject Use Case...Please help
Date Sun, 15 May 2016 19:23:24 GMT
Hi

Currently I am using flume for data ingestion and my use case as follows

Log repository--------log copy Script-------> Staging directory for  copied logs

Staging directory for  copied logs folder structure----Machine1log----a.log
                                                                                         
                                    -----b.log
                                                                                         
          Machine2log----a.log
                                                                                         
                                    -----b.log

Flume will copy these logs and replicate same structure in HDFS cluster. Beginning with which
is :
                                                                                         
      /user/hdfs/Machine1log----a.log
                                                                                         
                                                -----b.log
                                                                                         
                        Machine2log----a.log
                                                                                         
                                                  -----b.log


And creates 0 byte dummy file with same name so that Script wont copy the same log again as
it find 0 byte file already existing in source directory.


Can we do same things with apache nifi?

Keeping in mind two goals- same folder structure in HDFS and after moving file to HDFS it
should crete 0 byte dummy file in source directory.


Please help

Thanks,
Deepak




With Best Regards,
Deepak Tripathi
Philips Innovation campus
Bangalore-560045
[images]


________________________________
The information contained in this message may be confidential and legally protected under
applicable law. The message is intended solely for the addressee(s). If you are not the intended
recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction
of this message is strictly prohibited and may be unlawful. If you are not the intended recipient,
please contact the sender by return e-mail and destroy all copies of the original message.

Mime
View raw message