ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mimmo_c <mimmo.celano1...@gmail.com>
Subject Ignite DataStream Troughput
Date Fri, 14 Jul 2017 15:05:43 GMT

We're testing ignite DataStream troughput with a network of 1Gbps. If we
send a big file with scp we have a throughput of 150 Mb/s. 
We have 1 Ignite Server and 1 Ignite Client on version 2.0, in Client we
have an HashMap of 1Million Entry which we send into Server's cache. The
Cache is configured as Partitioned, Atomic, no backup. With a simple
DataStream when we send the HashMap into the cache we have a throughput of
20/25 Mb/s without Stream Transformer. Is this a limit of DataStream? 

Here is the code:

  try(IgniteDataStreamer<String, Long> stmr =
      for(Entry<String, Long> entry : mappa1.entrySet()){

When we add another Server Node the each Server Node receive 11 Mb/s
(effective throughput is the same 20/25 Mb/s)

If we Add a StreamTransformer for update value in cache and send more than 1
Hashmap the throughput is 15 Mb/s and if we add another Server Node the
throughput is the same.

public static class UpdateWord extends StreamTransformer<String, Long> {
        public Object process(MutableEntry<String, Long> e, Object... arg)
throws EntryProcessorException {
        	Object streamed = arg[0];
        	Long temp = Long.valueOf(streamed.toString()).longValue();
        	Long val = e.getValue();
        	e.setValue(val == null ? temp : val + temp);
            return null;

Is there some limit on DataStream Usage or a bottleneck? How can we  use all
network bandwidth? 
Maybe some DataStream Configuration?


View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-DataStream-Troughput-tp14918.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

View raw message