ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergi Vladykin <sergi.vlady...@gmail.com>
Subject Re: Inefficient approach to executing remote SQL queries
Date Tue, 23 May 2017 09:16:36 GMT
Michael,

I see your point. I think it must not be too hard to start asynchronously
establishing connections to all the needed nodes.

I've created respective issue in Jira:
https://issues.apache.org/jira/browse/IGNITE-5277

Sergi

2017-05-23 11:56 GMT+03:00 Michael Griggs <michael.griggs@gridgain.com>:

> Hi Val
>
> This is precisely my point: it's only a minor optimization until the point
> when establishing each connection takes 3-4 seconds, and we establish 32 of
> them in sequence.  At that point it becomes a serious issue: the customer
> cannot run SQL queries from their development machines without them timing
> out once out of every two or three runs.  These kind of problems undermine
> confidence in Ignite.
>
> Mike
>
>
> -----Original Message-----
> From: Valentin Kulichenko [mailto:valentin.kulichenko@gmail.com]
> Sent: 22 May 2017 19:15
> To: dev@ignite.apache.org
> Subject: Re: Inefficient approach to executing remote SQL queries
>
> Hi Mike,
>
> Generally, establishing connections in parallel could make sense, but note
> that in most this would be a minor optimization, because:
>
>    - Under load connections are established once and then reused. If you
>    observe disconnections during application lifetime under load, then
>    probably this should be addressed first.
>    - Actual communication is asynchronous, we use NIO for this. If
>    connection already exists, sendGeneric() basically just puts a message
> into
>    a queue.
>
> -Val
>
> On Mon, May 22, 2017 at 7:04 PM, Michael Griggs <
> michael.griggs@gridgain.com
> > wrote:
>
> > Hi Igniters,
> >
> >
> >
> > Whilst diagnosing a problem with a slow query, I became aware of a
> > potential issue in the Ignite codebase.  When executing a SQL query
> > that is to run remotely, the IgniteH2Indexing#send() method is called,
> > with a Collection<ClusterNode> as one of its parameters.  This
> > collection is iterated sequentially, and ctx.io().sendGeneric() is
> > called synchronously for each node.  This is inefficient if
> >
> >
> >
> > a)       This is the first execution of a query, and thus TCP connections
> > have to be established
> >
> > b)      The cost of establishing a TCP connection is high
> >
> >
> >
> > And optionally
> >
> >
> >
> > c)       There are a large number of nodes in the cluster
> >
> >
> >
> > In my current situation, developers want to run test queries from
> > their code running locally, but connected via VPN to their UAT server
> > environment.
> > The
> > cost of opening a TCP connection is in the multiple seconds, as you
> > can see from this Ignite log file snippet:
> >
> > 2017-05-22 18:29:48,908 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/7.1.14.242:56924,
> > rmtAddr=/10.132.80.3:47100]
> >
> > 2017-05-22 18:29:52,294 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/7.1.14.242:56923,
> > rmtAddr=/10.132.80.30:47102]
> >
> > 2017-05-22 18:29:58,659 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/7.1.14.242:56971,
> > rmtAddr=/10.132.80.23:47101]
> >
> > 2017-05-22 18:30:03,183 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/7.1.14.242:56972,
> > rmtAddr=/10.132.80.21:47100]
> >
> > 2017-05-22 18:30:06,039 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/7.1.14.242:56973,
> > rmtAddr=/10.132.80.21:47103]
> >
> > 2017-05-22 18:30:10,828 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/7.1.14.242:57020,
> > rmtAddr=/10.132.80.20:47100]
> >
> > 2017-05-22 18:30:13,060 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/7.1.14.242:57021,
> > rmtAddr=/10.132.80.29:47103]
> >
> > 2017-05-22 18:30:22,144 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/7.1.14.242:57022,
> > rmtAddr=/10.132.80.22:47103]
> >
> > 2017-05-22 18:30:26,513 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/7.1.14.242:57024,
> > rmtAddr=/10.132.80.20:47101]
> >
> > 2017-05-22 18:30:28,526 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/7.1.14.242:57025,
> > rmtAddr=/10.132.80.30:47103]
> >
> >
> >
> > Comparing the same code that is executed inside of the UAT environment
> > (so not using the VPN):
> >
> > 2017-05-22 18:22:18,102 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/10.175.11.38:53288,
> > rmtAddr=/10.175.11.58:47100]
> >
> > 2017-05-22 18:22:18,105 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/10.175.11.38:45890,
> > rmtAddr=/10.175.11.54:47101]
> >
> > 2017-05-22 18:22:18,108 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/127.0.0.1:47582,
> > rmtAddr=/127.0.0.1:47100]
> >
> > 2017-05-22 18:22:18,111 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/127.0.0.1:45240,
> > rmtAddr=/127.0.0.1:47103]
> >
> > 2017-05-22 18:22:18,114 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/10.175.11.38:46280,
> > rmtAddr=/10.175.11.15:47100]
> >
> > 2017-05-22 18:22:18,118 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/10.132.80.21:51476,
> > rmtAddr=/10.132.80.29:47103]
> >
> > 2017-05-22 18:22:18,120 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/10.132.80.21:56274,
> > rmtAddr=pocfd-master1/10.132.80.22:47103]
> >
> > 2017-05-22 18:22:18,124 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/10.132.80.21:53558,
> > rmtAddr=pocfd-ignite1/10.132.80.20:47101]
> >
> > 2017-05-22 18:22:18,127 INFO [TcpCommunicationSpi] - Established
> > outgoing communication connection [locAddr=/10.132.80.21:56216,
> > rmtAddr=/10.132.80.30:47103]
> >
> >
> >
> > This is a design flaw in the Ignite code, as we are relying on the
> > client's network behaving in a particular way (i.e., port opening being
> very fast).
> > We should instead try to mask this potential slowness by establishing
> > connections in parallel, and waiting on the results.
> >
> >
> >
> > I would like to hear others thoughts and comment before we open a JIRA
> > to look at this.
> >
> >
> >
> > Regards
> >
> > Mike
> >
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message