nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <joe.w...@gmail.com>
Subject Re: sFTP processor not working well when fetching big file
Date Wed, 17 May 2017 17:17:09 GMT
Hello

Can you please clarify the processor(s) you're using.  It is the
ListSftp and FetchSftp processors?  Or is it GetSFTP?

For the processor to send the data to the success relationship it
should mean that the transfer session reports a successful conclusion.
Can you show us the logs for the transfer that are in the
logs/nifi-app.log?

Thanks
Joe

On Wed, May 17, 2017 at 1:14 PM, Qinghao Dai <qdai@cloudera.com> wrote:
> Hi All,
>
> I am using sFTP processor to fetch some big file (larger then 3 or 5 Gb)
> from sFTP server.
>
> This fetching will continue for around 1 hour long due to network bandwidth.
>
> But the processor does not work well, it will only get like 90% or 95% part
> of the file.
>
> Thus it will get incomplete file for me. This may be due to network problem.
>
> But it put the file onto "success" relation and thinks it's a success.
> That's the part that's causing me problem.
>
> Did anybody here hit this problem too? Any solution or workaround?
>
> And what specific argument do I need to set for fetching big files.
>
> Thanks!
> --
> Qinghao Dai

Mime
View raw message