ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Evgenii Zhuravlev <e.zhuravlev...@gmail.com>
Subject Re: Index performance
Date Mon, 04 Dec 2017 14:41:02 GMT
Hi,

Please share benchmarks code, it's hard to understand why you can't load
CPU for 100% without your code.

But I see one thing, that definitely could lead to performance problems -
you've set  <property name="queryParallelism" value=“48"/> - this could
lead to a lot of context switching since every query could be executed in
48 threads. I'm sure that you run a lot of queries in parallel, so, it's a
very big value for this property, even if you have 48 CPU on a machine.

I would recommend using the default value for this property - 1. But, if
you want to run a few of long queries, it could be increased.

Also, this property doesn't work for SQL queries, and, actually, for a
default off-heap cache since version 2.0 at all:

<property name="copyOnRead" value="true"/>

Evgenii

2017-12-04 16:36 GMT+03:00 Ali <alimovalisher@gmail.com>:

> Hi!
>
> I found poor sql index performance and CPU utilisation in ignite 2.3
> version when try to benchmark cluster for performance.
>
> *Cluster configuration*:
>
> 6 x 2 E5-2600v2 256GB 10GB link
> 1 x 4 E7 node (edge) from which tests are running
>
> *Ignite config*
>
>
> <property name="cacheConfiguration">
>     <bean class="org.apache.ignite.configuration.CacheConfiguration">
>         <property name="name" value=“test"/>
>         <property name="backups" value="0"/>
>         <property name="atomicityMode" value="ATOMIC"/>
>         <property name="cacheMode" value="PARTITIONED"/>
>         <property name="copyOnRead" value="true"/>
>         <property name="readFromBackup" value="true"/>
>         <property name="queryParallelism" value=“48"/>
>         <property name="rebalanceMode" value="ASYNC"/>
>         <property name="statisticsEnabled" value="false"/>
>
>         <property name="queryEntities">
>             <list>
>                 <bean class="org.apache.ignite.cache.QueryEntity">
>
>                     <constructor-arg value="java.lang.String"/>
>                     <constructor-arg value=“com.example.Test"/>
>
>                     <property name="tableName" value=“Test"/>
>                     <property name="fields">
>                         <map>
>                             <entry key=“field-1" value="java.lang.String"/>
>                             <entry key=“field-2" value="java.lang.String"/>
>                             <entry key=“field-3" value="java.lang.String"/>
>                             <entry key=“field-4" value="java.lang.String"/>
>                             <entry key=“field-5" value="java.lang.String"/>
>                             <entry key=“field-6" value="java.lang.Long"/>
>                         </map>
>                     </property>
>
>                     <property name="indexes">
>                         <list>
>                             <bean class="org.apache.ignite.cache.QueryIndex">
>                                 <property name="name" value="fields_idx"/>
>                                 <property name="fields">
>                                     <map>
>                                         <entry key=“field-2" value="true"/>
>                                         <entry key=“field-3" value="true"/>
>                                         <entry key=“field-4" value="true"/>
>                                         <entry key=“field-5" value="true"/>
>                                         <entry key=“field-6" value="true"/>
>                                     </map>
>                                 </property>
>                                 <property name="inlineSize" value="24"/>
>                                 <property name="indexType" value="SORTED"/>
>                             </bean>
>                         </list>
>                     </property>
>                 </bean>
>             </list>
>         </property>
>     </bean>
> </property>
>
>
> *Query*
>
> public Collection<Test> find(String mcc, String mnc, String lac, String cid, long
lastSeenDt, long offset, long limit) {
>     SqlQuery<String, Test> qry = new SqlQuery<>(
>             Entry.class,
>             "select * from Test where field-2 = ? and field-3 = ? and field-4 = ? and
field-5 = ? and field-6 >= ? limit ? offset ?"
>     );
>
>     qry.setArgs(mcc, mnc, lac, cid, lastSeenDt, limit, offset);
>
>     try (QueryCursor<Cache.Entry<String, Test>> query = cache.query(qry))
{
>         return StreamSupport.stream(query.spliterator(), false)
>                             .map(Cache.Entry::getValue)
>                             .collect(Collectors.toList());
>     }
> }
>
>
> *Benchmark*
>
>
> Benchmark consist of several java processes that call this query through
> RMI  in {1..20 000} threads but maximum CPU utilisation doesn’t increase
> more than 9% on each node and doesn’t depend on load
>
> simple IgniteCache#get test fully utilise CPU
>
>
>
> With best regards
> Alisher Alimov
> alimovalisher@gmail.com
>
>

Mime
View raw message