flume-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Horrocks <ch...@hor.rocks>
Subject Re: Flume bechmarks
Date Fri, 14 Oct 2016 07:30:01 GMT
I've got a pretty well resourced pre-production environment that might do the trick quite nicely.


-- Chris Horrocks


On Thu, Oct 13, 2016 at 4:55 pm, Lior Zeno <'liorzino@gmail.com'> wrote:
I think that we can come up with an initial version with little efforts.
The simplest scenario I can think of is running a Flume instance (with a
SeqGen source and a Null sink) for one minute, and then report the average
events per second.

On Thu, Oct 13, 2016 at 6:43 PM, Attila Simon <sati@cloudera.com> wrote:

> Good idea! What would be required to set up something similar for Flume?
> ie initial time cost for setting up the infrastructure and periodic time
> cost to add new use-cases.
>
> Cheers,
> Attila
>
>
>
> On Thu, Oct 13, 2016 at 5:19 PM, Lior Zeno <liorzino@gmail.com> wrote:
>
> > Hi All,
> >
> > Monitoring Flume's performance over time is an important step in every
> > production-level application. Benchmarking Flume on a nightly basis has
> > the following advantages:
> >
> > * Better understanding of Flume's bottlenecks.
> > * Allow users to compare the performance of different solutions, such as
> > Logstash and Fluentd.
> > * Better understanding of the influence of recent commits on performance.
> >
> > Logstash already conducts various performance tests, more details in this
> > link:
> > http://logstash-benchmarks.elastic.co/
> >
> > I propose adding a few micro-benchmarks showing Flume's TPS vs date (of
> > course, in the ideal case where the input and/or output do not bottleneck
> > the system), e.g. using the SeqGen source.
> >
> > Thoughts?
> >
> > Thanks
> >
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message