nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dale.chang13" <dale.chan...@outlook.com>
Subject FetchFile Cannot Allocate Enough Memory
Date Fri, 29 Apr 2016 13:12:13 GMT
I have been trying to run my data flow and I have been running into a problem
with being unable to read FetchFiles. I will detail my process below and I
would like some confirmation of my suspicions.

First I am ingesting an initial file that is fairly large, which contains
the path/filename of a ton of text files within another directory. The goal
is to read in the content of that large file, then read in the contents of
the thousands of text files, and then store the text file content into Solr.

The problem I am having is that the second FetchFile, the one that reads in
the smaller text files, frequently reports an error: /FileNotFoundException
xxx.txt (Cannot allocate memory); routing to failure/. This FetchFile runs
for about 20000 files and then continuously reports the above error for the
rest of the files.

My suspicion is of two concerns: not enough heap space vs. not enough
content_repo/flowfile_repo space. Any ideas or questions?



--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/FetchFile-Cannot-Allocate-Enough-Memory-tp9720.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Mime
View raw message