impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henry Robinson <he...@apache.org>
Subject Re: SendersBlockedTimer
Date Thu, 25 May 2017 18:23:16 GMT
Hi Evo -

Just to clarify: the EXCHANGE_NODE is the operator in the plan tree which
mediates communication between workers, not between the client and the
coordinator.

The SendersBlockedTimer measures the amount of time that senders have row
batches to deliver to an exchange node, but the exchange is busy delivering
a previously sent row batch. That is, the senders are sending faster than
the exchange node (and the upstream plan) processes those rows.

In a select * from table query, there'll be one exchange on the
coordinator, but that's not generally true - exchanges connect all the
fragment instances. Having the senders blocked in this case is typical,
because there'll lots of senders sending at high rate fanning in to a
single receiver, serving a single client.

The delivery of rows to the client is managed by the coordinator fragment
instance through a different part of the code to the exchange node.

Henry

On 25 May 2017 at 08:31, Evo Eftimov <evo.eftimov@isecc.com> wrote:

> What is the purpose of SendersBlockedTimer attribute in the EXCHANGE_NODE
> section of the Coordinator Fragment – part of the PROFILE of SQL statement
> executed by Impala
>
>
>
> I have reviewed the Impala source code and know that the Exchange Node
> uses a Blocking Queue as part of “Stream Manager” module which it
> instantiates
>
>
>
> In the specific context I am interested in, the Exchange Node returns the
> row from a result set to a JDBC driver client. The result set is produced
> by a simple full table scan only query of the type “select * from table”
>
>
>
> The “Sender” Parallel Threads (presumably with the Exchange Node) publish
> rows to the Blocking Queue also in the Exchange Node and the JDBC client
> reads rows from the same queue via remote JDBC session / connection over
> TCP/IP – is that a correct description of how the Exchange Node mediates
> between JDBC client on the one hand and ImpalaD workers on the other? Btw
> the Exchange Node is part of the Coordinator Node in terms of terminology –
> right?
>
>
>
> My specific question is what is the purpose/meaning  of
>   SendersBlockedTimer – e.g. does it mean that the Sender Threads WITHIN
> the Exchange Node have been in a blocked state for the time shown in the
> value of the attribute. And if this is correct then does that mean that
> they have been blocked because the JDBC Client couldn’t not keep up with
> draining the Blocking Queue during the aggregated time duration in
> SendersBlockedTimer?
>
>
>

Mime
View raw message