nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject svn commit: r1689930 - /incubator/nifi/site/trunk/quickstart.html
Date Wed, 08 Jul 2015 18:23:55 GMT
Author: aldrin
Date: Wed Jul  8 18:23:55 2015
New Revision: 1689930

Adding Linux configuration information as well as linking to the administration guide within
the development quickstart.


Modified: incubator/nifi/site/trunk/quickstart.html
--- incubator/nifi/site/trunk/quickstart.html (original)
+++ incubator/nifi/site/trunk/quickstart.html Wed Jul  8 18:23:55 2015
@@ -109,6 +109,38 @@
 Then checkout the &#39;develop&#39; branch</p>
 <pre><code>git checkout develop</code></pre>
+<h3 id="linux-operating-system-configuration">Linux Operating System Configuration</h3>
+<p><em>NOTE</em>: If you are building on Linux, consider these best practices.
Typical Linux defaults are not necessarily well tuned for the needs of an IO intensive application
like NiFi. 
+For all of these areas, your distribution&#39;s requirements may vary.  Use these sections
as advice, but consult your distribution-specific documentation for how best to achieve these
+<h4 id="maximum-file-handles">Maximum File Handles</h4>
+<p>NiFi will at any one time potentially have a very large number of file handles open.
 Increase the limits by
+editing &#39;/etc/security/limits.conf&#39; to add something like</p>
+<pre><code>*  hard  nofile  50000
+*  soft  nofile  50000</code></pre>
+<h4 id="maximum-forked-processes">Maximum Forked Processes</h4>
+<p>NiFi may be configured to generate a significant number of threads.  To increase
the allowable number edit &#39;/etc/security/limits.conf&#39;</p>
+<pre><code>*  hard  nproc  10000
+*  soft  nproc  10000</code></pre>
+<p>And your distribution may require an edit to /etc/security/limits.d/90-nproc.conf
by adding</p>
+<pre><code>*  soft  nproc  10000</code></pre>
+<h4 id="increase-the-number-of-tcp-socket-ports-available">Increase the number of TCP
socket ports available</h4>
+<p>This is particularly important if your flow will be setting up and tearing down
a large number of sockets in small period of time.</p>
+<pre><code>sudo sysctl -w net.ipv4.ip_local_port_range=&quot;10000 65000&quot;</code></pre>
+<h4 id="set-how-long-sockets-stay-in-a-timed_wait-state-when-closed">Set how long sockets
stay in a TIMED_WAIT state when closed</h4>
+<p>You don&#39;t want your sockets to sit and linger too long given that you want
to be able to quickly setup and teardown new sockets.  It is a good idea to read more about
+it but to adjust do something like</p>
+<pre><code>sudo sysctl -w net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait=&quot;1&quot;</code></pre>
+<h4 id="tell-linux-you-never-want-nifi-to-swap">Tell Linux you never want NiFi to swap</h4>
+<p>Swapping is fantastic for some applications.  It isn&#39;t good for something
+NiFi that always wants to be running.  To tell Linux you&#39;d like swapping off you
+can edit &#39;/etc/sysctl.conf&#39; to add the following line</p>
+<pre><code>vm.swappiness = 0</code></pre>
+<h4 id="disable-partition-atime">Disable partition atime</h4>
+<p>For the partitions handling the various NiFi repos turn off things like &#39;atime&#39;.
+Doing so can cause a surprising bump in throughput.  Edit the &#39;/etc/fstab&#39;
+and for the partition(s) of interest add the &#39;noatime&#39; option.</p>
+<h4 id="additional-guidance">Additional guidance</h4>
+<p>Additional information on system administration and settings can be located in our
<a href="">Administrator&#39;s
 <h3 id="build-steps">Build steps</h3>
 <li>You need a recent Java 7 (or newer) JDK.</li>

View raw message