nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <>
Subject Re: Calculating the theoretical throughput of a Nifi server
Date Mon, 17 Oct 2016 11:08:48 GMT

I suspect bottlenecks in the software itself and the flow design will
become a factor before you 800 MB/s. You'd likely hit CPU efficiency
issues before this caused by the flow processors themselves and due to
garbage collection.  Probably the most important factor though will be
the transaction rate and whether the flow is configured to tradeoff
some latency for higher throughput.  So many variables at play but
under idealized conditions and a system like you describe it is
theoretically feasible to hit that value.

Practically speaking I think you'd be looking at a couple hundred MB/s
per server like this on real flows on a sustained basis.


On Sun, Oct 16, 2016 at 11:06 PM, Ali Nazemian <> wrote:
> Dear Nifi users/developers,
> Hi,
> I was wondering how can I calculate the theoretical throughput of a Nifi
> server? let's suppose we can eliminate different bottlenecks such as the
> file flow rep and provenance repo bottleneck by using a very high-end SSD.
> Moreover, assume that a very high-end network infrastructure is available.
> In this case, is it possible to reach 800MB throughput per second per each
> server? Suppose each server comes with 24 disk slots. 16 disk slots are used
> for creating 8 x RAID1(SAS 10k) mount points and are dedicated to the
> content repo. Let's say each content repo can achieve 100 MB throughput. May
> I say the total throughput per each server can be 8x100=800MBps?  Is it
> possible to reach this amount of throughput practically?
> Thank you very much.
> Best regards,
> Ali

View raw message