Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <CA+0=VoV-=E5xJqAMdxULRR2esbRO_XMis2nZJ4nDzf2jUev4Qg@mail.gmail.com>
References: <CA+eZwrFfzcwdV2SHxjVrFqFx5UHPxKScW8=8xHQ8B8rKbYq8LA@mail.gmail.com>
 <CANCAXEF2ue70S9Dbj2Na8xP6vvnkSVO6hQnc71H+cQeJok-1ag@mail.gmail.com>
 <CALOGAoyhsQ=h2POJbb1zyniN+51C_+bWVQQm1FF9M15H4h5Epw@mail.gmail.com>
 <CA+eZwrGMFV-AK5nyQJ+EXcK01s28BTWzwvW_-fg=h9mW034UBw@mail.gmail.com>
 <CANCAXEFJMWrEiKrEo6Uc_dYQUx0h9n0W-SZ90EniMrgVDk6nzA@mail.gmail.com>
 <CA+eZwrEvywn8Fy_1O=VOOBYu-8b0FEpEY6B6yf3FgArOvwJWDw@mail.gmail.com>
 <CANCAXEEu+tfx2YYv2r93FqNHspAo9t+7wR80jyDwWyaSxe161w@mail.gmail.com>
 <CA+eZwrGb9-fQWXbzSW5YKRy2uREyii0_pk4pfDjDog6EXXchEg@mail.gmail.com>
 <CANCAXEGBUZr+gmhsV+BBH65MZ6NSV63dWHXa8dzPz2Oc4M-O5Q@mail.gmail.com>
 <9EB58B66-8FE8-4B9C-9253-BBE7D5EF835D@apache.org> <CA+0=VoUgNo42YK7wUJmAXUHGhbvpFr5GoJ8VwURiOg+QZp5Kzg@mail.gmail.com>
 <7226F351-452F-4ECF-B228-FEB4EBF0A635@apache.org> <CA+0=VoV-=E5xJqAMdxULRR2esbRO_XMis2nZJ4nDzf2jUev4Qg@mail.gmail.com>
From: Sergi Vladykin <sergi.vladykin@gmail.com>
Date: Thu, 13 Apr 2017 09:34:21 +0300
Message-ID: <CA+eZwrGm1DiwkLecTHNf=_5PwP21FfBfw_vqxpF1vcBd3QuYWQ@mail.gmail.com>
Subject: Re: SQL on PARTITIONED vs REPLICATED cache
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary=94eb2c1a2d6c53f4c2054d0686f8
archived-at: Thu, 13 Apr 2017 06:35:09 -0000

--94eb2c1a2d6c53f4c2054d0686f8
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Yes, it is a correct explanation. I've created the issue

https://issues.apache.org/jira/browse/IGNITE-4955

Sergi

2017-04-13 0:25 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:

> Got it, Denis. I think you are right.
>
> On Wed, Apr 12, 2017 at 2:20 PM, Denis Magda <dmagda@apache.org> wrote:
>
> > Dmitriy,
> >
> > No, I think that Sergi supposed a type of cache which reference is used
> > for a query execution. In my example
> >
> > >> 2. Error-prone scenario - *replicatedCache*.query(=E2=80=9CSELECT * =
FROM
> > >> partitionedCache =E2=80=A6 JOIN replicatedCache =E2=80=A6=E2=80=9D);
> >
> > *replicatedCache* reference is used for the query execution and, as I
> > understand, this causes the issue.
> >
> > Sergi, please clarify.
> >
> > =E2=80=94
> > Denis
> >
> > > On Apr 12, 2017, at 1:51 PM, Dmitriy Setrakyan <dsetrakyan@apache.org=
>
> > wrote:
> > >
> > > Denis, I think that you meant selecting from replicated cache first a=
s
> an
> > > invalid scenario, but provided the wrong example. Here is the correct
> > > example for the invalid query:
> > >
> > > SELECT * FROM replicatedCache =E2=80=A6 JOIN partitionedCache =E2=80=
=A6=E2=80=9D
> > >
> > > I do agree, we should make the change, as long as we keep the flag to
> > > enable the old behavior.
> > >
> > > D.
> > >
> > > On Wed, Apr 12, 2017 at 12:50 PM, Denis Magda <dmagda@apache.org>
> wrote:
> > >
> > >> Sergi,
> > >>
> > >> As far as I understand you=E2=80=99re considering an example below:
> > >>
> > >> IgniteCache partitioneCache =3D ...;
> > >> IgniteCache replicatedCache =3D =E2=80=A6;
> > >>
> > >> 1. Valid scenario - *partitionedCache*.query(=E2=80=9CSELECT * FROM
> > >> partitionedCache =E2=80=A6 JOIN replicatedCache =E2=80=A6=E2=80=9D);
> > >> 2. Error-prone scenario - *replicatedCache*.query(=E2=80=9CSELECT * =
FROM
> > >> partitionedCache =E2=80=A6 JOIN replicatedCache =E2=80=A6=E2=80=9D);
> > >>
> > >> Do you mean 2. as the issue? If it=E2=80=99s so then can=E2=80=99t w=
e just detect on
> our
> > >> own that all the caches are replicated and execute a query more
> optimal?
> > >> This should omit necessity to add isReplicatedOnly()?
> > >>
> > >> =E2=80=94
> > >> Denis
> > >>
> > >>> On Apr 12, 2017, at 7:07 AM, Andrey Mashenkov <
> > >> andrey.mashenkov@gmail.com> wrote:
> > >>>
> > >>> Yes, it's reasonable.
> > >>>
> > >>> On Wed, Apr 12, 2017 at 3:23 PM, Sergi Vladykin <
> > >> sergi.vladykin@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> Good point, but I'm not sure. The difference is that on client nod=
e
> > you
> > >>>> should not be able to enable isLocal, while isReplicatedOnly is
> > >> perfectly
> > >>>> valid. What do you think?
> > >>>>
> > >>>> Sergi
> > >>>>
> > >>>> 2017-04-12 15:18 GMT+03:00 Andrey Mashenkov <
> > andrey.mashenkov@gmail.com
> > >>> :
> > >>>>
> > >>>>> Sergi,
> > >>>>>
> > >>>>> Got it.
> > >>>>>
> > >>>>> Does query execution way and results will be same for
> > isReplicatedOnly
> > >>>> flag
> > >>>>> and for isLocal flag turned on?
> > >>>>> If my understanding is correct, we will get same results and ther=
e
> is
> > >> no
> > >>>>> need to introduce a new flag.
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Wed, Apr 12, 2017 at 2:54 PM, Sergi Vladykin <
> > >>>> sergi.vladykin@gmail.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Ok, let it be an exception. I'm just saying that the thing does
> not
> > >>>> work
> > >>>>>> now.
> > >>>>>>
> > >>>>>> Sergi
> > >>>>>>
> > >>>>>> 2017-04-12 14:50 GMT+03:00 Andrey Mashenkov <
> > >>>> andrey.mashenkov@gmail.com
> > >>>>>> :
> > >>>>>>
> > >>>>>>> Sergi,
> > >>>>>>>
> > >>>>>>> I wounder how it is possible?
> > >>>>>>>
> > >>>>>>> Looks like it is impossible to run query on replicated cache, b=
ut
> > >>>>> select
> > >>>>>>> data from a
> > >>>>>>> partitioned table. It will result with IlleagalStateException o=
n
> > >>>> stable
> > >>>>>>> topology or
> > >>>>>>> IgniteCacheException on unstable topology.
> > >>>>>>> See ReduceQueryExecutor.stableDataNodes() and
> > >>>>>>> replicatedUnstableDataNodes()
> > >>>>>>> methods.
> > >>>>>>>
> > >>>>>>> BTW, IlleagalStateException with no message is confusing.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Wed, Apr 12, 2017 at 2:36 PM, Sergi Vladykin <
> > >>>>>> sergi.vladykin@gmail.com>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Andrey,
> > >>>>>>>>
> > >>>>>>>> Because if you run query on replicated cache, but select data
> from
> > >>>> a
> > >>>>>>>> partitioned table, you will get only a part of the result.
> > >>>>>>>>
> > >>>>>>>> Igor,
> > >>>>>>>>
> > >>>>>>>> You are mostly right, but
> > >>>>>>>>
> > >>>>>>>> 1. Performance characteristics may change.
> > >>>>>>>> 2. Ignite SQL processing pipeline may not support all the stuf=
f
> in
> > >>>> H2
> > >>>>>> SQL
> > >>>>>>>> and fail in some case where it worked previously.
> > >>>>>>>>
> > >>>>>>>> Because of this the change may affect existing applications an=
d
> I
> > >>>>> want
> > >>>>>> to
> > >>>>>>>> have it in 2.0 to make it legal.
> > >>>>>>>>
> > >>>>>>>> Sergi
> > >>>>>>>>
> > >>>>>>>> 2017-04-12 14:10 GMT+03:00 Igor Sapego <isapego@gridgain.com>:
> > >>>>>>>>
> > >>>>>>>>> Also, is it really a breaking change if the results are wrong=
?
> > >>>>>>>>> To me it looks more like a bugfix, i.e. you can't break
> something
> > >>>>>>>>> that does not work properly.
> > >>>>>>>>>
> > >>>>>>>>> Best Regards,
> > >>>>>>>>> Igor
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Apr 12, 2017 at 2:04 PM, Andrey Mashenkov <
> > >>>>>>>>> andrey.mashenkov@gmail.com> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Sergi,
> > >>>>>>>>>>
> > >>>>>>>>>> How can query to replicated cache leads to to wrong results?
> > >>>>>>>>>> Is it due to we can read backup entries?
> > >>>>>>>>>>
> > >>>>>>>>>> On Wed, Apr 12, 2017 at 12:31 PM, Sergi Vladykin <
> > >>>>>>>>> sergi.vladykin@gmail.com
> > >>>>>>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Guys,
> > >>>>>>>>>>>
> > >>>>>>>>>>> I want to introduce another breaking change for 2.0.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Currently SQL is being processed differently when we call
> > >>>>> method
> > >>>>>>>>> `query`
> > >>>>>>>>>> on
> > >>>>>>>>>>> partitioned cache and on replicated: on replicated cache we
> > >>>> do
> > >>>>>> not
> > >>>>>>> do
> > >>>>>>>>> any
> > >>>>>>>>>>> extra processing and execute the query as is on current nod=
e.
> > >>>>>>>>>>>
> > >>>>>>>>>>> This behavior historically existed for performance reasons.
> > >>>> But
> > >>>>>> it
> > >>>>>>> is
> > >>>>>>>>> not
> > >>>>>>>>>>> obvious and leads to wrong query results. This issue become=
s
> > >>>>> even
> > >>>>>>>> more
> > >>>>>>>>>>> creepy with JDBC and ODBC drivers.
> > >>>>>>>>>>>
> > >>>>>>>>>>> In 2.0 I want to execute all the SQL queries the same way
> > >>>>> through
> > >>>>>>> the
> > >>>>>>>>>> whole
> > >>>>>>>>>>> processing pipeline to guaranty the correct result
> > >>>>> irrespectively
> > >>>>>>> to
> > >>>>>>>>> the
> > >>>>>>>>>>> cache that was the query originator.
> > >>>>>>>>>>>
> > >>>>>>>>>>> To be able to have the old behavior (skip all the
> > >>>> preprocessing
> > >>>>>> and
> > >>>>>>>> run
> > >>>>>>>>>>> query on current node) add a flag isReplicatedOnly() on
> > >>>>> SqlQuery
> > >>>>>>> and
> > >>>>>>>>>>> SqlFieldsQuery. It will be disabled by default and if one
> > >>>> knows
> > >>>>>>> that
> > >>>>>>>>> the
> > >>>>>>>>>>> only replicated tables participate in a query, then he can
> > >>>>> enable
> > >>>>>>> it
> > >>>>>>>>> for
> > >>>>>>>>>>> better performance.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Sergi
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> --
> > >>>>>>>>>> Best regards,
> > >>>>>>>>>> Andrey V. Mashenkov
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> Best regards,
> > >>>>>>> Andrey V. Mashenkov
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Best regards,
> > >>>>> Andrey V. Mashenkov
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Best regards,
> > >>> Andrey V. Mashenkov
> > >>
> > >>
> >
> >
>

--94eb2c1a2d6c53f4c2054d0686f8--