ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Syed Moizuddin <moiz.gro...@gmail.com>
Subject AffinityKey for co-location
Date Fri, 29 Jan 2016 11:27:15 GMT
Hi,

My use case is to optimize certain aggregation queries in a very large
and flat table.
The table schema would be like below:

EmployeeID , DepartmentID, <Employee Details ...> < Department Details
...> <Company details >

Most of my queries are "group by" over <EmployeeID, DepartmentID>, and
nested as:

select ... ( select ... group by EmployeeID ) ... group by EmployeeID,
DepartmentID

>From  my basic Ignite understanding, keeping the EmployeeID,
DepartmentID co-located would help in query performance, assuming a
EmpoyeeID is always linked to a single  DepartmentID, I tried the
following:

1. created AffinityKey<EmployeeID, DepartmentID> in the pojo
public class Model implements Serializable{

@QuerySqlField(index = true)
private String employeeID;

@QuerySqlField(index = true)
private String departmentID;

...
public AffinityKey<String> key() {
    if (key == null)
        key = new AffinityKey<>(employeeID, employeeID);
    return key;
}
...
}

2. used DataStreamer  to load cache as follow (inspired by
CacheQueryExample.initialize):

CacheConfiguration<AffinityKey<String>, Model> cacheConfiguration =
new CacheConfiguration<>(CACHE_NAME);

 cacheConfiguration.setCacheMode(CacheMode.PARTITIONED);
 cacheConfiguration.setIndexedTypes(AffinityKey.class, Model.class);
 try (IgniteCache<AffinityKey<String>, Model> cache =
ignite.getOrCreateCache(cacheConfiguration)) {
       try (IgniteDataStreamer<AffinityKey<String>, Model> stmr =
ignite.dataStreamer(CACHE_NAME)) {
       ...
       Model model = new Model();
       ...
       stmr.addData(model.key(), model);
       ...

I am doing it wrong here as I get only one record per key (EmployeeID,
DepartmentID) in the cache when I load the data.

Could you please let me know the correct usage of the AffinityKey to
achieve co-location , or if I need to try out other ways for it.

Regards,
Moiz
PS: ... used to hide some relevant code

Mime
View raw message