nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Jackoway <al...@cloudera.com>
Subject Re: Content Repository Cleanup
Date Sat, 10 Dec 2016 02:55:49 GMT
One other note on this, when it came back up there were tons of messages
like this:

2016-12-09 18:36:36,244 INFO [main] o.a.n.c.repository.FileSystemRepository
Found unknown file /path/to/content_repository/498/1481329796415-87538
(1071114 bytes) in File System Repository; archiving file

I haven't dug into what that means.
Alan

On Fri, Dec 9, 2016 at 9:53 PM, Alan Jackoway <alanj@cloudera.com> wrote:

> Hello,
>
> We have a node on which nifi content repository keeps growing to use 100%
> of the disk. It's a relatively high-volume process. It chewed through more
> than 100GB in the three hours between when we first saw it hit 100% of the
> disk and when we just cleaned it up again.
>
> We are running nifi 1.1 for this. Our nifi.properties looked like this:
>
> nifi.content.repository.implementation=org.apache.
> nifi.controller.repository.FileSystemRepository
> nifi.content.claim.max.appendable.size=10 MB
> nifi.content.claim.max.flow.files=100
> nifi.content.repository.directory.default=./content_repository
> nifi.content.repository.archive.max.retention.period=12 hours
> nifi.content.repository.archive.max.usage.percentage=50%
> nifi.content.repository.archive.enabled=true
> nifi.content.repository.always.sync=false
>
> I just bumped retention period down to 2 hours, but should max usage
> percentage protect us from using 100% of the disk?
>
> Unfortunately we didn't get jstacks on either failure. If it hits 100%
> again I will make sure to get that.
>
> Thanks,
> Alan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message