nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Payne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-949) Allow configuration of cleanup resources
Date Mon, 14 Sep 2015 19:02:46 GMT

    [ https://issues.apache.org/jira/browse/NIFI-949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14744044#comment-14744044
] 

Mark Payne commented on NIFI-949:
---------------------------------

I will be interested to see if this is still a problem after 0.3.0. We made some really big
improvements in smarts behind the Content Repository so that it generates far fewer files
on disk. This also translates to far less cleanup needed. There were also performance improvements
in how the Provenance Repository ages off old data, whereby it avoids ever even updating the
Lucene Index (which was about 98% of the time required for cleanup).

In order to see the difference, though, you'd have to run 0.3.0 for long enough for any data
that you may have archived in the Content Repo to be aged off. After that, though, please
let us know if you are still having problems with this.

Thanks
-Mark

> Allow configuration of cleanup resources
> ----------------------------------------
>
>                 Key: NIFI-949
>                 URL: https://issues.apache.org/jira/browse/NIFI-949
>             Project: Apache NiFi
>          Issue Type: Improvement
>            Reporter: Brandon DeVries
>            Priority: Minor
>
> Allow the allocation of resource to the cleanup of the content and provenance repositories
to be configurable.   There are times at which the resources spent performing cleanup activities
can degrade the operation of the flow sufficiently to cause problems.  Sometimes this requires
removing the offending node from the cluster to allow it to clean up without taking on new
data (and thus backing up).  Making the cleanup resources configurable could prevent such
a situation by allowing them to be reduced under times of heavy load to keep the node useable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message