ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Kuznetsov <akuznet...@gridgain.com>
Subject Re: cache cannot load all the data into cache
Date Wed, 20 Apr 2016 09:47:49 GMT
Hi, Kevin.

Could you please make small example + db scripts to reproduce and debug
this issue?


On Wed, Apr 13, 2016 at 10:34 AM, Zhengqingzheng <zhengqingzheng@huawei.com>
wrote:

> Dear all,
> I am trying to load two tables data into caches to speed up my queries.
> table1 contains 564 records, with one primary key as index.
> definition of table content from java  as follows:
>     @QuerySqlField
>     private String orgId;
>
>     @QuerySqlField(index=true)
>     private String objId;
>
>     @QuerySqlField
>     private int numRows;
>
> table2 contains 9626 records, with no primary key defined but a group
> index is defined.
> definition of table2 from java as follows:
>     @QuerySqlField
>     private String orgId;
>
>     @QuerySqlField(orderedGroups={@QuerySqlField.Group(
>             name="objId_fieldName_idx", order=0, descending = true)})
>     private String objId;
>
>     @QuerySqlField(orderedGroups={@QuerySqlField.Group(
>             name="objId_fieldName_idx", order=1, descending = true)})
>     private String fieldName;
>
>     @QuerySqlField
>     private int fieldNum;
>
>     @QuerySqlField
>     private int statVal;
>
> I defined two caches to load all the data from two tables:
> the first cache load data from table1, and works fine.
> but the second cache which load data from table2 cannot load all the data,
> only few of them.
> I think this is due to the configuration of cache2 was probability wrong[
> because cache content shows that objid was the unique key to retrieve the
> data record]:
>
>                 final String CACHE_NAME1=
> IgniteMetaDatabaseFieldStat.class.getSimpleName()+"_Cache";
>         CacheConfiguration<String, IgniteMetaDatabaseFieldStat> cfg =
>                 new CacheConfiguration<String,
> IgniteMetaDatabaseFieldStat>(CACHE_NAME1);
>
>         CacheJdbcPojoStoreExampleFactory<String,
> IgniteMetaDatabaseFieldStat> storeFactory =
>                 new
> CacheJdbcPojoStoreExampleFactory<String,IgniteMetaDatabaseFieldStat>();
>
>         storeFactory.setDialect(new OracleDialect());
>
>         JdbcType jdbcType = new JdbcType();
>
>         jdbcType.setCacheName(CACHE_NAME1);
>         jdbcType.setDatabaseSchema("besdb");
>         jdbcType.setDatabaseTable("data_base_field_stat");
>
> <b>
>         jdbcType.setKeyType("java.lang.String");
>         jdbcType.setKeyFields(new JdbcTypeField(Types.VARCHAR, "OBJID",
> String.class, "objId")
>            /* ,new JdbcTypeField(Types.VARCHAR, "FIELDNAME", String.class,
> "fieldName")*/);
> </b>
>
> jdbcType.setValueType("org.apache.ignite.examples.model.IgniteMetaDatabaseFieldStat");
>         jdbcType.setValueFields(
>                 new JdbcTypeField(Types.VARCHAR,"ORGID", String.class,
> "orgId"),
>                 new JdbcTypeField(Types.VARCHAR,"OBJID", String.class,
> "objId"),
>                 new JdbcTypeField(Types.VARCHAR,"FIELDNAME", String.class,
> "fieldName"),
>                 new JdbcTypeField(Types.INTEGER,"FIELDNUM", Integer.class,
> "fieldNum"),
>                 new JdbcTypeField(Types.INTEGER,"STAT_VAL", Integer.class,
> "statVal")
>                     );
>
>
>         storeFactory.setTypes(jdbcType);
>
>         cfg.setCacheStoreFactory(storeFactory);
>
>         // Set atomicity as transaction, since we are showing transactions
> in the example.
>         cfg.setAtomicityMode(CacheAtomicityMode.ATOMIC);
>         cfg.setIndexedTypes(String.class,
> IgniteMetaDatabaseFieldStat.class);
>
>         cfg.setReadThrough(true);
>         cfg.setWriteThrough(true);
>
>         cfg.setCacheMode(CacheMode.PARTITIONED);
>         //cfg.setAtomicityMode(CacheAtomicityMode.ATOMIC);
>         //cfg.setMemoryMode(CacheMemoryMode.OFFHEAP_TIERED);
>         //cfg.setOffHeapMaxMemory(64 * 1024L * 1024L);
>
>         //cfg.setStartSize(100 * 1024 * 1024);
>         cfg.setBackups(0);
>
> please note the jdbcType.setKeyTypes and setKeyFields part. I want to use
> the <b>groupIndex</b> as the cache key setting, which was defined in the
> annotation part of
>  @QuerySqlField(orderedGroups={@QuerySqlField.Group(
>             name="objId_fieldName_idx", order=0, descending = true)})
>     private String objId;
>
>     @QuerySqlField(orderedGroups={@QuerySqlField.Group(
>             name="objId_fieldName_idx", order=1, descending = true)})
>     private String fieldName;
>
> but I don't know how to do that, in my example I just use objid as the
> key. In this case, if there are duplicate values come into the cache, the
> rest values was ignored.
>
> How to setup the correct key for jdbcTypes in my cache configuration?
>
>
> Best regards,
> Kevin
>
>


-- 
Alexey Kuznetsov
GridGain Systems
www.gridgain.com

Mime
View raw message