ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Roger Fischer (CW)" <rfis...@Brocade.com>
Subject Affinity and String key (was: SQLQuery with simple Join return no results)
Date Tue, 05 Sep 2017 16:44:47 GMT
Hi Denis,

I don’t quite understand your comment “If you use Strings as the keys you won’t get
affinity collocation set up properly”.

I have an object with a plain String key, named switchId, and another object with a composite
key, of which one field is switchId (of type String). I am using switchId as the affinity
key, and it seems to work fine without distributed queries. Was this a coincident and I should
re-test?

My understanding is that caches with no explicit affinity key use the key to distribute objects.
For the plain-key cache this would be switchId.

So, if I have two caches, both with switchId as the key, I should get automatic collocation
(no setup required). Correct?

And for the third cache, with the switchId field in the composite key as the affinity key,
I should also get collocation (with the other two caches). Correct?

What am I missing?

BTW, I think there was a typo in the response (first sentence), and you meant to say “set
up properly and _non_-distributed joins will return an incomplete result.”.

Roger

PS: My configuration (all XML):

        <property name="cacheKeyConfiguration">
            <list>
                <!-- ports cache -->
                <bean class="org.apache.ignite.cache.CacheKeyConfiguration">
                    <property name="typeName" value="com.abc.poc.inppoc.model.PortKey"/>
                    <property name="affinityKeyFieldName" value="switchId"/>
                </bean>
                <!-- switches cache uses default affinity key (id) -->
            </list>
        </property>



                <!-- ports cache -->
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="PortCache"/>
                    <property name="cacheMode" value="PARTITIONED"/>
                    <property name="backups" value="1"/>

                    <!-- use affinity (on Switch ID) -->
                    <property name="affinity">
                        <bean class="org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction">
                            <property name="partitions" value="1024"/>
                        </bean>
                    </property>

                    <!-- Port fields/columns -->
                    <property name="queryEntities">
                        <list>
                            <bean class="org.apache.ignite.cache.QueryEntity">
                                <property name="keyType" value="com.abc.poc.inppoc.model.PortKey"/>
                                <property name="valueType" value="com.abc.poc.inppoc.model.Port"/>
                                <property name="fields">
                                    …


                <!-- switches cache -->
                <bean class="org.apache.ignite.configuration.CacheConfiguration">
                    <property name="name" value="SwitchCache"/>
                    <property name="cacheMode" value="PARTITIONED"/>
                    <property name="backups" value="1"/>

                    <!-- use affinity (on Switch ID, ie primary key) -->
                    <property name="affinity">
                        <bean class="org.apache.ignite.cache.affinity.rendezvous.RendezvousAffinityFunction">
                            <property name="partitions" value="1024"/>
                        </bean>
                    </property>

                    <!-- Switch fields/columns -->
                    <property name="queryEntities">
                        <list>
                            <bean class="org.apache.ignite.cache.QueryEntity">
                                <property name="keyType" value="java.util.UUID"/>
                                <property name="valueType" value="com.brocade.poc.inppoc.model.Switch"/>
                                <property name="fields">
                                    …

PortKey.java:

public class PortKey implements Serializable {

    private UUID    id;        // port-id; PK
    private UUID    switchId;  // affinity key; not really part of PK

    …



From: Denis Magda [mailto:dmagda@apache.org]
Sent: Friday, September 01, 2017 3:07 PM
To: user@ignite.apache.org
Subject: Re: SQLQuery with simple Join return no results

If you use Strings as the keys you won’t get affinity collocation set up properly and distributed
joins will return an incomplete result. One of the keys have to comprise a “parent” class
key that will be an affinity key. Look at the example here:
https://apacheignite.readme.io/docs/affinity-collocation#section-collocate-data-with-data<https://urldefense.proofpoint.com/v2/url?u=https-3A__apacheignite.readme.io_docs_affinity-2Dcollocation-23section-2Dcollocate-2Ddata-2Dwith-2Ddata&d=DwMFaQ&c=IL_XqQWOjubgfqINi2jTzg&r=1esZO0r0bYS90lcsaLA6N4AFxuNo6lzauhETGwdJQoQ&m=KTlx1Z0HYXpMSZKonCG-plYFaijMpCt0OUS3tJgKbWQ&s=-YW5SFh-ko4-_EBwKFMKH5BOJU7pGcQMqaCRZrwZz24&e=>

As for the NON collocated joins suggested by Roger (qry.setDistributedJoins( true)), I would
use them only if it’s impossible to set up the collocation between 2 entities. That’s
not your case from what I see. NON collocated joins are slower than collocated ones.

—
Denis


On Sep 1, 2017, at 2:53 PM, Roger Fischer (CW) <rfische@Brocade.com<mailto:rfische@Brocade.com>>
wrote:

Hi Matt,

are the objects to join collocated, ie. do they have the same affinity key? If yes, it should
work (it worked for me).

If no, you need to enable distributed joins for the query. See the middle line.

           SqlFieldsQuery qry = new SqlFieldsQuery( stmt);
           qry.setDistributedJoins( true);
           queryCursor = aCache.query( qry);

Roger

-----Original Message-----
From: matt [mailto:goodieboy@gmail.com]
Sent: Friday, September 01, 2017 1:52 PM
To: user@ignite.apache.org<mailto:user@ignite.apache.org>
Subject: SQLQuery with simple Join return no results

I have 2 caches defined, both with String keys, and classes that make use of the Ignite annotations
for indexes and affinity. I've got 3 different nodes running, and the code I'm using to populate
the cache w/test data works, and I can see each node is updated with its share of the data.
My index types are set on the caches as well.

If I do a ScanQuery, I can see that all of the fields and IDs are correct, Ignite returns
them all. But when doing a SqlQuery, I get nothing back.
Ignite is not complaining about the query, it's just returning an empty cursor.

If I remove the Join, results are returned.

So I'm wondering if this is related to the way I've set up my affinity mapping. It's basically
setup like the code below... and the query looks like this:

"from B, A WHERE B.id = A.bID"

Any ideas on what I'm doing wrong here?

class A implements Serializable {
 @QuerySqlField(index = true)
 String id;

 @QuerySqlField(index = true)
 String bId;

 @AffinityKeyMapped
 @QuerySqlField(index = true)
 String group;
}

class B implements Serializable {
 @QuerySqlField(index = true)
 String id;

 @AffinityKeyMapped
 @QuerySqlField(index = true)
 String group;
}



--
Sent from: https://urldefense.proofpoint.com/v2/url?u=http-3A__apache-2Dignite-2Dusers.70518.x6.nabble.com_&d=DwICAg&c=IL_XqQWOjubgfqINi2jTzg&r=1esZO0r0bYS90lcsaLA6N4AFxuNo6lzauhETGwdJQoQ&m=g9B7o3WZd7LuW39MRFWdU5Gim1S3DRPZGcugI0n9Wno&s=ehBogkdblpG8U9N0taPx5Jdf8G9uDMnZiRR-f34rEe0&e=

Mime
View raw message