[ https://issues.apache.org/jira/browse/MINIFI-425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16329114#comment-16329114
]
Saulo Sobreiro commented on MINIFI-425:
---------------------------------------
[~aldrin] by the time of the log, I would say that all those restarts happened when I was
trying to apply the configurations through nifi.properties, so yes, I think those are my fault.
"_If not, my suspicion is that you were encountering a heap issue from the SplitText. Typically
this is aided by tiering SplitText processors (split a very large file into 1k chunks and
then do individual splits)._"
Indeed I had that issue but as you can find in the config.yml I implemented the solution you
are suggesting :). Difference is that my first SplitText is configured for 10k instead of
1k. Can this difference create a problem?
I attached a new log file minifi-bootstrap_2018-01-10.log from the day I had, and hopefully
fixed this issue. I hope it can provide furthers insights regarding the problem.
Indeed the minifi-app.log is lite, but I guess it is because it just has logs for the current
day... Looking for older logs I found only errors related to disk space like (minifi-app.log
for 15/01):
{code:java}
2018-01-15 21:52:13,560 INFO [pool-31-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating
checkpoint of FlowFile Repo
sitory
2018-01-15 21:52:13,685 ERROR [pool-31-thread-1] org.wali.MinimalLockingWriteAheadLog Failed
to create new journal for [Parti
tion-0, java.io.IOException: No space left on device] due to {}
java.io.IOException: No space left on device
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:326)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at org.wali.MinimalLockingWriteAheadLog$Partition.rollover(MinimalLockingWriteAheadLog.java:788)
at org.wali.MinimalLockingWriteAheadLog.checkpoint(MinimalLockingWriteAheadLog.java:528)
at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.checkpoint(WriteAheadFlowFileRepository.java:45
1)
at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository$1.run(WriteAheadFlowFileRepository.java:423)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:1
80)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFuture
{code}
Every other log looks like the one I attached.
> MiNiFi contents_repository config problem
> -----------------------------------------
>
> Key: MINIFI-425
> URL: https://issues.apache.org/jira/browse/MINIFI-425
> Project: Apache NiFi MiNiFi
> Issue Type: Bug
> Affects Versions: 0.3.0
> Reporter: Saulo Sobreiro
> Priority: Major
> Attachments: config.yml, minifi-app.log, minifi-bootstrap.log, minifi-bootstrap_2018-01-10.log
>
>
> *Problem:*
> I found 2 problems in this my MiNiFi-NiFi setup:
> - The first is related to my MiNiFi ./content_repository. This path is using
> around 120GB of disk space, which is far more than I can afford for this
> application;
> - The second is related to the solution I tried to the previous problem. I
> tried to configure the MiNiFi nifi.properties with the following values:
> ´´´
> nifi.content.repository.archive.max.retention.period=6 hours
> nifi.content.repository.archive.max.usage.percentage=5%
> ´´´
> However, when I restart MiNiFi this instance for it to use the new
> properties, I noticed that it overwrites my nifi.properties file with some
> default values erasing the new values.
>
> *Versions in use:*
> NiFi - version 1.2.0 installed with HDF (3.0.2.0-76)
> MiNiFi - version 0.3.0 (transfered from
> [http://mirrors.up.pt/pub/apache/nifi/minifi/0.3.0/minifi-0.3.0-bin.tar.gz])
> NiFi OS - RHEL 7.4
> MiNiFi OS - RHEL 7.2
>
> *config.yml*
> The config.yml file can be found attached. It is the version used in production after
replacing some information like URLs.
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
|