nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Saulo Sobreiro (JIRA)" <>
Subject [jira] [Commented] (MINIFI-425) MiNiFi contents_repository config problem
Date Wed, 17 Jan 2018 18:01:07 GMT


Saulo Sobreiro commented on MINIFI-425:

[~aldrin] by the time of the log, I would say that all those restarts happened when I was
trying to apply the configurations through, so yes, I think those are my fault.


"_If not, my suspicion is that you were encountering a heap issue from the SplitText.  Typically
this is aided by tiering SplitText processors (split a very large file into 1k chunks and
then do individual splits)._"

Indeed I had that issue but as you can find in the config.yml I implemented the solution you
are suggesting :). Difference is that my first SplitText is configured for 10k instead of
1k. Can this difference create a problem?

I attached a new log file minifi-bootstrap_2018-01-10.log from the day I had, and hopefully
fixed this issue. I hope it can provide furthers insights regarding the problem.


Indeed the minifi-app.log is lite, but I guess it is because it just has logs for the current
day... Looking for older logs I found only errors related to disk space like (minifi-app.log
for 15/01):
2018-01-15 21:52:13,560 INFO [pool-31-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating
checkpoint of FlowFile Repo
2018-01-15 21:52:13,685 ERROR [pool-31-thread-1] org.wali.MinimalLockingWriteAheadLog Failed
to create new journal for [Parti
tion-0, No space left on device] due to {} No space left on device
at Method)
at org.wali.MinimalLockingWriteAheadLog$Partition.rollover(
at org.wali.MinimalLockingWriteAheadLog.checkpoint(
at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.checkpoint(
at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository$
at java.util.concurrent.Executors$
at java.util.concurrent.FutureTask.runAndReset(
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFuture
Every other log looks like the one I attached.



> MiNiFi contents_repository config problem
> -----------------------------------------
>                 Key: MINIFI-425
>                 URL:
>             Project: Apache NiFi MiNiFi
>          Issue Type: Bug
>    Affects Versions: 0.3.0
>            Reporter: Saulo Sobreiro
>            Priority: Major
>         Attachments: config.yml, minifi-app.log, minifi-bootstrap.log, minifi-bootstrap_2018-01-10.log
> *Problem:*
> I found 2 problems in this my MiNiFi-NiFi setup:
> - The first is related to my MiNiFi ./content_repository. This path is using 
> around 120GB of disk space, which is far more than I can afford for this 
> application; 
> - The second is related to the solution I tried to the previous problem. I 
> tried to configure the MiNiFi with the following values: 
> ´´´ 
> nifi.content.repository.archive.max.retention.period=6 hours 
> nifi.content.repository.archive.max.usage.percentage=5% 
> ´´´ 
> However, when I restart MiNiFi this instance for it to use the new 
> properties, I noticed that it overwrites my file with some 
> default values erasing the new values. 
> *Versions in use:*
> NiFi - version 1.2.0 installed with HDF ( 
> MiNiFi - version 0.3.0 (transfered from 
> []) 
> NiFi OS - RHEL 7.4
> MiNiFi OS - RHEL 7.2 
> *config.yml*
> The config.yml file can be found attached. It is the version used in production after
replacing some information like URLs. 

This message was sent by Atlassian JIRA

View raw message