ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yakov Zhdanov <yzhda...@apache.org>
Subject Re: Performance question
Date Tue, 29 Nov 2016 08:41:09 GMT
I think it is a bad idea to set number of threads greater than CPU count
given you do not block on any IO operations

However, benchmark itself seems totally incorrect to me. The most important
issues are
1. total time is too low. try to put it to a loop body and measure each
iterations. you can also (moreover, I would like you to) use jmh for local
operations benchmarks
2. you don't have any warm up phase.

Please refer to
http://stackoverflow.com/questions/504103/how-do-i-write-a-correct-micro-benchmark-in-java
or any other resource explaining on how to create benchmarks.

--Yakov

2016-11-29 15:14 GMT+07:00 Vladislav Pyatkov <vldpyatkov@gmail.com>:

> Hi Alisher,
>
> It look doubt for me. You parallelize the job, but got a performance
> decrease.
> I recommend to use a java profiler and try to separate long time methods.
>
> How are you get a list of local partition (can it contain excess numbers)?
> And please check, has forckjoin pool enough size:
>
> -Djava.util.concurrent.ForkJoinPool.common.parallelism=1024
>
>
> On Nov 28, 2016 8:39 PM, "Alisher Alimov" <alimovalisher@gmail.com> wrote:
>
>> I found only one way to parallelize read via ScanQuery
>>
>> int[] partitions = this.ignite.affinity("test.cache").primaryPartitions(this.ignite.cluster().node());
>>
>> startTime = System.currentTimeMillis();
>>
>> Arrays.stream(partitions)
>>       .parallel()
>>       .forEach(partition -> {
>>           ScanQuery<Object, Object> qry = new ScanQuery<>(partition);
>>           qry.setLocal(true);
>>           qry.setPageSize(5_000);
>>
>>           QueryCursor<Cache.Entry<Object, Object>> query = cache.query(qry);
>>           List<Cache.Entry<Object, Object>> all = query.getAll();
>>       });
>>
>> System.out.println(String.format("Complete in: %dms", System.currentTimeMillis()
- startTime));
>>
>>
>> But it’s doesn’t help a lot (speed was downgrade on 10-20%) or there is
>> another good solution to do it?
>>
>>
>>
>> With best regards
>> Alisher Alimov
>> alimovalisher@gmail.com
>>
>>
>>
>>
>> On 28 Nov 2016, at 19:38, Alexey Goncharuk <alexey.goncharuk@gmail.com>
>> wrote:
>>
>> Hi Alisher,
>>
>> As Nicolae suggested, try parallelizing your scan using per-partition
>> iterator. This should give you almost linear performance growth up to the
>> number of available CPUs.
>> Also make sure to set CacheConfiguration#copyOnRead flag to false.
>>
>> --AG
>>
>> 2016-11-28 19:31 GMT+03:00 Marasoiu Nicolae <Nicolae.Marasoiu@cegeka.com>
>> :
>>
>>> ​Regarding CPU load, a single thread of execution exists in the program
>>> so (at most) one core is used. So if you have 8 cores, it means that it is
>>> 8 to 16 times slower than a program able to use all the cores & CPU
>>> redundancy of the machine.
>>>
>>> In my tests, indeed, a core looks fully utilized. To me, scanning 1M
>>> key-values per second is pretty ok, but indeed, if LMAX got 6M transactions
>>> per core per second, it can perhaps go up, but something tells me this will
>>> not be the limitation of the typical application.
>>>
>>>
>>> Met vriendelijke groeten / Meilleures salutations / Best regards
>>>
>>> *Nicolae Marasoiu*
>>> *Agile Developer*
>>>
>>> *E*  *Nicolae.Marasoiu@cegeka.com <Nicolae.Marasoiu@cegeka.com>*
>>>
>>> CEGEKA 15-17 Ion Mihalache Blvd. Tower Center Building,
>>> 4th,5th,6th,8th,9th fl
>>> RO-011171 Bucharest (RO), Romania
>>> *T* +40 21 336 20 65
>>> *WWW.CEGEKA.COM <http://www.cegeka.com/>*  [image: LinkedIn]
>>> <https://www.linkedin.com/company/cegeka-romania>
>>> ------------------------------
>>> *De la:* Alisher Alimov <alimovalisher@gmail.com>
>>> *Trimis:* 28 noiembrie 2016 15:27
>>> *Către:* user@ignite.apache.org
>>> *Subiect:* Performance question
>>>
>>> Hello!
>>>
>>> I have write and run a simple performance test to check
>>> IgniteCache#localEntries and found that current method is not enough fast.
>>>
>>> Ignite ignite = Ignition.start();
>>>
>>>
>>> CacheConfiguration<UUID, UUID> cacheConfiguration = new CacheConfiguration<>();
>>> cacheConfiguration.setBackups(0);
>>>
>>> IgniteCache<UUID, UUID> cache = ignite.getOrCreateCache("test.cache");
>>>
>>> for (int i = 0; i < 1_000_000; i++) {
>>>     cache.put(UUID.randomUUID(), UUID.randomUUID());
>>> }
>>>
>>> long startTime = System.currentTimeMillis();
>>>
>>> cache.localEntries(CachePeekMode.PRIMARY).forEach(entry -> {
>>> });
>>>
>>> System.out.println(String.format("Complete in: %dms", System.currentTimeMillis()
- startTime));
>>>
>>>
>>> Reading local entries take about 1s (1000 rows per ms) that’s is low.
>>> Test was run on server with provided configuration with default Ignite
>>> configs, load average was about 0 and CPU was not busy more than 10%
>>> Intel(R) Xeon(R) CPU           E5645  @ 2.40GHz
>>>
>>>
>>> May be I do  or configure something wrong or current speed is normal?
>>>
>>>
>>> With best regards
>>> Alisher Alimov
>>> alimovalisher@gmail.com
>>>
>>>
>>>
>>>
>>>
>>
>>

Mime
View raw message