ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: proposed design for thin client SQL management and monitoring (view running queries and kill it)
Date Wed, 14 Nov 2018 10:25:25 GMT
Yury,

As I understand you mean that the view should contains both running and
> finished queries. If be honest for the view I was going to use just queries
> running right now. For finished queries I thought about another view with
> another set of fields which should include I/O related ones. Is it works?


Got you, so if only running queries are there then your initial proposal
makes total sense. Not sure we need a view of the finished queries. It will
be possible to analyze them through the updated DetailedMetrics approach,
won't it?

For "KILL QUERY node_id query_id"  node_id required as part of unique key
> of query and help understand Ignite which node start the distributed query.
> Use both parameters will allow cheap generate unique key across all nodes.
> Node which started a query can cancel it on all nodes participate nodes.
> So, to stop any queries initially we need just send the cancel request to
> node who started the query. This mechanism is already in Ignite.


Can we locate node_id behind the scenes if the user supplies query_id only?
A query record in the view already contains query_id and node_id and it
sounds like an extra work for the user to fill in all the details for us.
Embed node_id into query_id if you'd like to avoid extra network hops for
query_id to node_id mapping.

--
Denis

On Wed, Nov 14, 2018 at 1:04 AM Юрий <jury.gerzhedowich@gmail.com> wrote:

> Denis,
>
> Under the hood 'time' will be as startTime, but for system view I planned
> use duration which will be simple calculated as now - startTime. So, there
> is't a performance issue.
> As I understand you mean that the view should contains both running and
> finished queries. If be honest for the view I was going to use just queries
> running right now. For finished queries I thought about another view with
> another set of fields which should include I/O related ones. Is it works?
>
> For "KILL QUERY node_id query_id"  node_id required as part of unique key
> of query and help understand Ignite which node start the distributed query.
> Use both parameters will allow cheap generate unique key across all nodes.
> Node which started a query can cancel it on all nodes participate nodes.
> So, to stop any queries initially we need just send the cancel request to
> node who started the query. This mechanism is already in Ignite.
>
> Native SQL APIs will automatically support the futures after implementing
> for thin clients. So we are good here.
>
>
>
> вт, 13 нояб. 2018 г. в 18:52, Denis Magda <dmagda@apache.org>:
>
> > Yury,
> >
> > Please consider the following:
> >
> >    - If we record the duration instead of startTime, then the former has
> to
> >    be updated frequently - sounds like a performance red flag. Should we
> > store
> >    startTime and endTime instead? This way a query record will be updated
> >    twice - when the query is started and terminated.
> >    - In the IEP you've mentioned I/O related fields that should help to
> >    grasp why a query runs that slow. Should they be stored in this view?
> >    - "KILL QUERY query_id" is more than enough. Let's not add "node_id"
> >    unless it's absolutely required. Our queries are distributed and
> > executed
> >    across several nodes that's why the node_id parameter is redundant.
> >    - This API needs to be supported across all our interfaces. We can
> start
> >    with JDBC/ODBC and thin clients and then support for the native SQL
> APIs
> >    (Java, Net, C++)
> >    - Please share examples of SELECTs in the IEP that would show how to
> >    find long running queries, queries that cause a lot of I/O troubles.
> >
> > --
> > Denis
> >
> > On Tue, Nov 13, 2018 at 1:15 AM Юрий <jury.gerzhedowich@gmail.com>
> wrote:
> >
> > > Igniters,
> > >
> > > Some comments for my original email's.
> > >
> > > The proposal related to part of IEP-29
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-29%3A+SQL+management+and+monitoring
> > > >
> > > .
> > >
> > > What purpose are we pursuing of the proposal?
> > > We want to be able check which queries running right now through thin
> > > clients. Get some information related to the queries and be able to
> > cancel
> > > a query if it required for some reasons.
> > > So, we need interface to get a running queries. For the goal we propose
> > > running_queries system view. The view contains unique query identifier
> > > which need to pass to kill query command to cancel the query.
> > >
> > > What do you think about fields of the running queries view? May be some
> > > useful fields we could easy add to the view.
> > >
> > > Also let's discuss syntax of cancellation of query. I propose to use
> > MySQL
> > > like syntax as easy to understand and shorter then Oracle and Postgres
> > > syntax ( detailed information in IEP-29
> > > <
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-29%3A+SQL+management+and+monitoring
> > > >
> > > ).
> > >
> > >
> > >
> > > пн, 12 нояб. 2018 г. в 19:28, Юрий <jury.gerzhedowich@gmail.com>:
> > >
> > > > Igniters,
> > > >
> > > > Below is a proposed design for thin client SQL management and
> > monitoring
> > > > to cancel a queries.
> > > >
> > > > 1) Ignite expose system SQL view with name *running_queries*
> > > > proposed columns: *node_id, query_id, sql, schema_name,
> connection_id,
> > > > duration*.
> > > >
> > > > node_id - initial node of request
> > > > query_id - unique id of query on node
> > > > sql - text of query
> > > > schema name - name of sql schema
> > > > connection_id - id of client connection from
> > > ClientListenerConnectionContext
> > > > class
> > > > duration - duration in millisecond from start of query
> > > >
> > > >
> > > > Ignite will gather info about running queries from each of nodes and
> > > > collect it during user query. We already have most of the information
> > at
> > > GridRunningQueryInfo
> > > > on each of nodes.
> > > >
> > > > Instead of duration we can use start_time, but I think duration will
> be
> > > > simple to use due to it not depend on a timezone.
> > > >
> > > >
> > > > 2) Propose to use following syntax to kill a running query:
> > > >
> > > > *KILL QUERY node_Id query_id*
> > > >
> > > >
> > > > Both parameters node_id and query_id can be get through
> running_queries
> > > > system view.
> > > >
> > > > When a node receive such request it can be run locally in case node
> > have
> > > > given node_id or send message to node with given id. Because node
> have
> > > > information about local running queries then can cancel it - it
> already
> > > > implemented in GridReduceQueryExecutor.cancelQueries(qryId) method.
> > > >
> > > > Comments are welcome.
> > > > --
> > > > Живи с улыбкой! :D
> > > >
> > >
> > >
> > > --
> > > Живи с улыбкой! :D
> > >
> >
>
>
> --
> Живи с улыбкой! :D
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message