ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Paschenko <alexander.a.pasche...@gmail.com>
Subject Re: Communication from JDBC/ODBC drivers
Date Tue, 20 Dec 2016 18:18:45 GMT
OK, will remove redundant query tasks creation for the case of DML
statements - they at least incur some GC overhead even if not context
switching.

- Alex

2016-12-20 12:49 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:
> On Tue, Dec 20, 2016 at 1:23 AM, Alexander Paschenko <
> alexander.a.paschenko@gmail.com> wrote:
>
>> I see what you are suggesting and will remove this feature from the driver
>> if there are no objections. (Anyone?)
>> Still, for clarity and your better understanding I'd like to again point
>> out that there's no thread context switching or any Ignite imposed overhead
>> by default as even ignite compute ia not used — closure's call() method is
>> called directly in the same thread. Wrapped task isn't going anywhere in
>> local case. And it's the pattern that is used in Ignite code quite
>> extensively — if the task is local anyway, let's just run it "on the spot",
>> directly, without any unnecessary overhead.
>>
>>
> I disagree, there is an overhead, especially for relatively small batches.
> Why do you keep insisting on having this flawed design? Why not fix it
> properly by calling putAll(...) directly?
>
>
>> — Alex
>> 19 дек. 2016 г. 11:44 PM пользователь "Dmitriy Setrakyan" <
>> dsetrakyan@apache.org> написал:
>>
>> > Alex,
>> >
>> > What Val and I are suggesting is that we get rid of any JdbcCallable
>> calls
>> > and invoke putAll(...) directly. Wrapping it into JdbcCallable will slow
>> us
>> > down and introduce unnecessary thread context switch.
>> >
>> > Do you agree?
>> >
>> > D.
>> >
>> > On Mon, Dec 19, 2016 at 11:01 AM, Valentin Kulichenko <
>> > valentin.kulichenko@gmail.com> wrote:
>> >
>> > > By "locally executed callable" I meant something like this:
>> > >
>> > > new JdbcCallable(...).call();
>> > >
>> > > So it's not a network call and it's not even going through
>> IgniteCompute.
>> > > It's just the logic is incapsulated in the callable to support remote
>> > > execution of the same logic.
>> > >
>> > > But anyway, +1 to removing nodeId property. It doesn't make much sense
>> > with
>> > > the new implementation of JDBC.
>> > >
>> > > -Val
>> > >
>> > > On Mon, Dec 19, 2016 at 10:15 AM, Alexander Paschenko <
>> > > alexander.a.paschenko@gmail.com> wrote:
>> > >
>> > > > When node is local, no network interaction occurs on query send.
>> Would
>> > be
>> > > > shame otherwise :)
>> > > >
>> > > > — Alex
>> > > > 19 дек. 2016 г. 8:36 PM пользователь "Denis Magda"
<
>> dmagda@apache.org>
>> > > > написал:
>> > > >
>> > > > > Dmitriy,
>> > > > >
>> > > > > According to Val and Alex P. explanations this happens when a
>> > specific
>> > > > > node id is set. I got confused by the code flow initially.
>> > > > >
>> > > > > —
>> > > > > Denis
>> > > > >
>> > > > > > On Dec 19, 2016, at 9:13 AM, Dmitriy Setrakyan <
>> > > dsetrakyan@apache.org>
>> > > > > wrote:
>> > > > > >
>> > > > > > Thanks, Alex.
>> > > > > >
>> > > > > > Can you clarify if the putAll(...) call is made directly,
or from
>> > > > locally
>> > > > > > executed callable? According to Denis, "even single (non
batched)
>> > > > updates
>> > > > > > or queries are sent as callables", which should be fixed
in my
>> > view.
>> > > > > >
>> > > > > > D.
>> > > > > >
>> > > > > > On Mon, Dec 19, 2016 at 3:32 AM, Alexander Paschenko <
>> > > > > > alexander.a.paschenko@gmail.com> wrote:
>> > > > > >
>> > > > > >> Dima, Val,
>> > > > > >>
>> > > > > >> Introduction of updates has not changed public API at
all (only
>> > > > > >> configuration, in some cases), so they work in this
respect
>> > exactly
>> > > > > >> like SELECTs - by default they run on client node started
by the
>> > > > > >> driver itself, but can be sent via the same callables
mechanism
>> to
>> > > any
>> > > > > >> remote node by its id.
>> > > > > >>
>> > > > > >> So Dima, you're right, currently it's possible to send
query to
>> > any
>> > > > > >> given node. And, at the same time, currently by default
>> everything
>> > > > > >> works exactly like you want it to - that is, any MERGE,
batched
>> or
>> > > > > >> not, boils down to putAll, and by default this call
happens
>> > locally.
>> > > > > >>
>> > > > > >> - Alex
>> > > > > >>
>> > > > > >> 2016-12-17 18:47 GMT+03:00 Dmitriy Setrakyan <
>> > dsetrakyan@apache.org
>> > > >:
>> > > > > >>> On Fri, Dec 16, 2016 at 9:53 PM, Valentin Kulichenko
<
>> > > > > >>> valentin.kulichenko@gmail.com> wrote:
>> > > > > >>>
>> > > > > >>>> I'm not sure about updates, but can tell about
how selects are
>> > > > > >> implemented
>> > > > > >>>> there. Basically, there is an option to execute
the query on a
>> > > > > >> particular
>> > > > > >>>> node specified by ignite.jdbc.nodeId property.
Not sure why we
>> > > need
>> > > > > this
>> > > > > >>>> though, probably it's just leftover from the
legacy version of
>> > the
>> > > > > >> driver
>> > > > > >>>> based on thin client.
>> > > > > >>>>
>> > > > > >>>> If the property is set, the callable is sent
to a remote node.
>> > But
>> > > > if
>> > > > > >> it is
>> > > > > >>>> not, the same callable is created, but it is
invoked directly
>> on
>> > > the
>> > > > > >>>> embedded client which is the behavior that you
expect. And
>> it's
>> > > the
>> > > > > >> default
>> > > > > >>>> one.
>> > > > > >>>>
>> > > > > >>>>
>> > > > > >>> Ouch. If this is the reason, I would drop the nodeId
property.
>> I
>> > > > don't
>> > > > > >>> think it makes sense and it significantly slows
down the
>> > > > > implementation.
>> > > > > >>>
>> > > > > >>>
>> > > > > >>>> -Val
>> > > > > >>>>
>> > > > > >>>> On Fri, Dec 16, 2016 at 7:51 PM, Denis Magda
<
>> dmagda@apache.org
>> > >
>> > > > > wrote:
>> > > > > >>>>
>> > > > > >>>>> Frankly speaking, even single (non batched)
updates or
>> queries
>> > > are
>> > > > > >> sent
>> > > > > >>>> as
>> > > > > >>>>> callables. This is what I see in the code.
>> > > > > >>>>> No idea what was the reason behind this
design.
>> > > > > >>>>>
>> > > > > >>>>> Andrey G., Alex P. could you shed a light
on this?
>> > > > > >>>>>
>> > > > > >>>>> —
>> > > > > >>>>> Denis
>> > > > > >>>>>
>> > > > > >>>>>> On Dec 16, 2016, at 3:08 PM, Dmitriy
Setrakyan <
>> > > > > >> dsetrakyan@apache.org>
>> > > > > >>>>> wrote:
>> > > > > >>>>>>
>> > > > > >>>>>> To my understanding, we are implementing
JDBC batches by
>> > > sending a
>> > > > > >>>>> callable
>> > > > > >>>>>> to another node. If we already have
a client node on the
>> JDBC
>> > > > driver
>> > > > > >>>>> side,
>> > > > > >>>>>> why not just issue a putAll(...) call
from the client?
>> > > > > >>>>>>
>> > > > > >>>>>> D.
>> > > > > >>>>>>
>> > > > > >>>>>> On Fri, Dec 16, 2016 at 3:02 PM, Denis
Magda <
>> > dmagda@apache.org
>> > > >
>> > > > > >>>> wrote:
>> > > > > >>>>>>
>> > > > > >>>>>>> Dmitriy,
>> > > > > >>>>>>>
>> > > > > >>>>>>> JDBC drivers spawns an Ignite client
node and uses it for
>> > > cluster
>> > > > > >>>>>>> connectivity and queries execution.
Queries issued over the
>> > > JDBC
>> > > > > >> are
>> > > > > >>>>> turned
>> > > > > >>>>>>> into SqlFieldsQueries and sent to
the cluster in this form.
>> > > > > >>>>>>>
>> > > > > >>>>>>> ODBC driver works in a bit different
way. It connects to
>> the
>> > > > > >> cluster
>> > > > > >>>> via
>> > > > > >>>>>>> ODBC processor that needs to be
running on one of the
>> nodes:
>> > > > > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster-
>> > > > > >> configuration
>> > > > > >>>> <
>> > > > > >>>>>>> https://apacheignite.readme.io/docs/odbc-driver#cluster-
>> > > > > >> configuration
>> > > > > >>>>>
>> > > > > >>>>>>>
>> > > > > >>>>>>> —
>> > > > > >>>>>>> Denis
>> > > > > >>>>>>>
>> > > > > >>>>>>>> On Dec 16, 2016, at 2:41 PM,
Dmitriy Setrakyan <
>> > > > > >>>> dsetrakyan@apache.org>
>> > > > > >>>>>>> wrote:
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> Igniters,
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> Can someone explain to me how
Ignite executes SQL from
>> JDBC
>> > > and
>> > > > > >> ODBC
>> > > > > >>>>>>>> drivers? Do we start an Ignite
client node on the driver
>> > side?
>> > > > Or
>> > > > > >> do
>> > > > > >>>> we
>> > > > > >>>>>>> use
>> > > > > >>>>>>>> some other protocol to send
commands to one of the Ignite
>> > > nodes?
>> > > > > >>>>>>>>
>> > > > > >>>>>>>> Thanks,
>> > > > > >>>>>>>> D.
>> > > > > >>>>>>>
>> > > > > >>>>>>>
>> > > > > >>>>>
>> > > > > >>>>>
>> > > > > >>>>
>> > > > > >>
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>

Mime
View raw message