ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alisher Alimov <alimovalis...@gmail.com>
Subject Re: Performance question
Date Mon, 28 Nov 2016 17:38:59 GMT
I found only one way to parallelize read via ScanQuery

int[] partitions = this.ignite.affinity("test.cache").primaryPartitions(this.ignite.cluster().node());

startTime = System.currentTimeMillis();

Arrays.stream(partitions)
      .parallel()
      .forEach(partition -> {
          ScanQuery<Object, Object> qry = new ScanQuery<>(partition);
          qry.setLocal(true);
          qry.setPageSize(5_000);

          QueryCursor<Cache.Entry<Object, Object>> query = cache.query(qry);
          List<Cache.Entry<Object, Object>> all = query.getAll();
      });

System.out.println(String.format("Complete in: %dms", System.currentTimeMillis() - startTime));

But it’s doesn’t help a lot (speed was downgrade on 10-20%) or there is another good solution
to do it?



With best regards
Alisher Alimov
alimovalisher@gmail.com




> On 28 Nov 2016, at 19:38, Alexey Goncharuk <alexey.goncharuk@gmail.com> wrote:
> 
> Hi Alisher,
> 
> As Nicolae suggested, try parallelizing your scan using per-partition iterator. This
should give you almost linear performance growth up to the number of available CPUs.
> Also make sure to set CacheConfiguration#copyOnRead flag to false. 
> 
> --AG
> 
> 2016-11-28 19:31 GMT+03:00 Marasoiu Nicolae <Nicolae.Marasoiu@cegeka.com <mailto:Nicolae.Marasoiu@cegeka.com>>:
> ​Regarding CPU load, a single thread of execution exists in the program so (at most)
one core is used. So if you have 8 cores, it means that it is 8 to 16 times slower than a
program able to use all the cores & CPU redundancy of the machine.
> In my tests, indeed, a core looks fully utilized. To me, scanning 1M key-values per second
is pretty ok, but indeed, if LMAX got 6M transactions per core per second, it can perhaps
go up, but something tells me this will not be the limitation of the typical application.
> 
> Met vriendelijke groeten / Meilleures salutations / Best regards
> 
> Nicolae Marasoiu 
> Agile Developer
>   
> E  Nicolae.Marasoiu@cegeka.com <mailto:Nicolae.Marasoiu@cegeka.com>  
> 
> 	CEGEKA 15-17 Ion Mihalache Blvd. Tower Center Building, 4th,5th,6th,8th,9th fl
> RO-011171 Bucharest (RO), Romania
> T +40 21 336 20 65
> WWW.CEGEKA.COM <http://www.cegeka.com/> 	 <https://www.linkedin.com/company/cegeka-romania>

> De la: Alisher Alimov <alimovalisher@gmail.com <mailto:alimovalisher@gmail.com>>
> Trimis: 28 noiembrie 2016 15:27
> Către: user@ignite.apache.org <mailto:user@ignite.apache.org>
> Subiect: Performance question
>  
> Hello!
> 
> I have write and run a simple performance test to check IgniteCache#localEntries and
found that current method is not enough fast.
> 
> Ignite ignite = Ignition.start();
> 
> CacheConfiguration<UUID, UUID> cacheConfiguration = new CacheConfiguration<>();
> cacheConfiguration.setBackups(0);
> 
> IgniteCache<UUID, UUID> cache = ignite.getOrCreateCache("test.cache");
> 
> for (int i = 0; i < 1_000_000; i++) {
>     cache.put(UUID.randomUUID(), UUID.randomUUID());
> }
> 
> long startTime = System.currentTimeMillis();
> 
> cache.localEntries(CachePeekMode.PRIMARY).forEach(entry -> {
> });
> 
> System.out.println(String.format("Complete in: %dms", System.currentTimeMillis() - startTime));
> 
> Reading local entries take about 1s (1000 rows per ms) that’s is low. 
> Test was run on server with provided configuration with default Ignite configs, load
average was about 0 and CPU was not busy more than 10%
> Intel(R) Xeon(R) CPU           E5645  @ 2.40GHz
> 
> 
> May be I do  or configure something wrong or current speed is normal?
> 
> 
> With best regards
> Alisher Alimov
> alimovalisher@gmail.com <mailto:alimovalisher@gmail.com>
> 
> 
> 
> 
> 


Mime
View raw message