ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Mekhanikov <dmekhani...@gmail.com>
Subject Re: Register Contentious query quit slow
Date Tue, 05 Dec 2017 08:27:22 GMT
Jin,

> Fact is after I commented out IQ, RF and keep local listeners exists, no more
meta data update happened and the program still work.

I already explained you in the previous letter, where this metadata update
comes from. It's because you transfer IQ and RF over network.
On the other hand, local listener doesn't need to be transferred, so it
doesn't trigger metadata update.

Consider the following example:

public class QueryRegisterExample {
    public static void main(String[] args) {
        Ignition.setClientMode(true);

        try (Ignite ignite = Ignition.start("config/ignite.xml")) {

            IgniteCache<Object, BinaryObject> cache =
ignite.getOrCreateCache("cache");

            ContinuousQuery<Object, BinaryObject> qry = new ContinuousQuery<>();

            qry.setInitialQuery(new ScanQuery<>((k, v) -> true)); //
Registering lambda as initial query.

            qry.setLocalListener((evts) -> {
                int num = 0;
                for (CacheEntryEvent<?, ? extends BinaryObject> evt : evts)
                    num++;

                System.out.println("Processed " + num + " events");
            });

            cache.query(qry);

            System.out.println("Query registration finished");
        }
    }
}

As you can see, there is a lambda registered as initial query. When I run
this code, it causes metadata of 2 classes to be registered, because the
entire QueryRegisterExample class got serialized. I can't even tell, what
this lambda looks like in serialized format, because anything can be
captured into it and you won't even know.
Anonymous classes are just the same thing, I used lambdas to save some
space.

Serialization of anonymous classes is even not recommended by Oracle:
https://docs.oracle.com/javase/tutorial/java/javaOO/nested.html#serialization

It gets better if you change it the following way:

public class QueryRegisterExample {
    public static void main(String[] args) {
        Ignition.setClientMode(true);

        try (Ignite ignite = Ignition.start("config/ignite.xml")) {

            IgniteCache<Object, BinaryObject> cache =
ignite.getOrCreateCache("cache");

            ContinuousQuery<Object, BinaryObject> qry = new ContinuousQuery<>();

            qry.setInitialQuery(new ScanQuery<>(new
ExampleInitialQuery())); // Registering static class as initial query.

            qry.setLocalListener((evts) -> {
                int num = 0;
                for (CacheEntryEvent<?, ? extends BinaryObject> evt : evts)
                    num++;

                System.out.println("Processed " + num + " events");
            });

            cache.query(qry);

            System.out.println("Query registration finished");
        }
    }

    private static class ExampleInitialQuery implements
IgniteBiPredicate<Object, BinaryObject> {
        @Override public boolean apply(Object k, BinaryObject v) {
            return true;
        }
    }
}

In this case metadata of only one class is registered.

So, do the same thing in your code to avoid redundant objects to be sent
over network.

Denis

вт, 5 дек. 2017 г. в 3:24, gunman524 <guojin0524@126.com>:

> Denis,
>
> I can understand the meta data need to be updated when the CQ has initial
> query(IQ) or remote filter(RF). But my understading is only the object
> within IQ and RF need to be updated and the stuff in Local listeners does
> not.
>
> In my case, the logic in IQ and RF is simple string compare
>           e.getValue().<String> field("FROM_RDC")).equals(localRDC)
> and those can be update to cluster if necessary and does make scene.  But
> the object in local listener and all object it referenced does not need to
> be update to cluster.
>
> Fact is after I commented out IQ, RF and keep local listeners exists, no
> more meta data update happened and the program still work. Seems it proved
> my guess.
>
> I didn't go through Ignite code carefully for this part, but it feels like
> current logic will check whether IQ or RF is exists. If yes, will update
> the
> meta data of whole CQ instead of just IQ and RF part.
>
> Can we do this enhancement that do not update meta data in Local listener
> to
> cluster?  Or is there has technical issue?
>
> Thanks,
>
> Jin
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Mime
View raw message