Mailing-List: contact user-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@ignite.apache.org
Date: Fri, 14 Jul 2017 08:05:43 -0700 (MST)
From: mimmo_c <mimmo.celano1993@gmail.com>
To: user@ignite.apache.org
Message-ID: <1500044743596-14918.post@n6.nabble.com>
Subject: Ignite DataStream Troughput
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
archived-at: Fri, 14 Jul 2017 15:05:48 -0000

Hi,

We're testing ignite DataStream troughput with a network of 1Gbps. If we
send a big file with scp we have a throughput of 150 Mb/s. 
We have 1 Ignite Server and 1 Ignite Client on version 2.0, in Client we
have an HashMap of 1Million Entry which we send into Server's cache. The
Cache is configured as Partitioned, Atomic, no backup. With a simple
DataStream when we send the HashMap into the cache we have a throughput of
20/25 Mb/s without Stream Transformer. Is this a limit of DataStream? 


Here is the code:

  try(IgniteDataStreamer<String, Long> stmr =
ignite.dataStreamer("default")){
        	
      for(Entry<String, Long> entry : mappa1.entrySet()){
        	stmr.addData(entry.getKey(),entry.getValue());
        }
        	
  }  


When we add another Server Node the each Server Node receive 11 Mb/s
(effective throughput is the same 20/25 Mb/s)

If we Add a StreamTransformer for update value in cache and send more than 1
Hashmap the throughput is 15 Mb/s and if we add another Server Node the
throughput is the same.


public static class UpdateWord extends StreamTransformer<String, Long> {
    	
        @Override
        public Object process(MutableEntry<String, Long> e, Object... arg)
throws EntryProcessorException {
        	Object streamed = arg[0];
        	Long temp = Long.valueOf(streamed.toString()).longValue();
        	Long val = e.getValue();
        	e.setValue(val == null ? temp : val + temp);
            return null;
        }
    }


Is there some limit on DataStream Usage or a bottleneck? How can we  use all
network bandwidth? 
Maybe some DataStream Configuration?

Thanks,
Mimmo


--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-DataStream-Troughput-tp14918.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.