impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcel Kornacker (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-4014: Introduce query-wide execution state.
Date Wed, 30 Nov 2016 02:11:43 GMT
Marcel Kornacker has posted comments on this change.

Change subject: IMPALA-4014: Introduce query-wide execution state.

Patch Set 4:

File be/src/runtime/

PS4, Line 502:  QueryState* qs = ExecEnv::GetInstance()->query_exec_mgr()->GetQueryState(query_id_);
> ScopedQueryState ?
that's a bug: you need to keep the reference until the query completes. see the call to ReleaseQueryState()
in TearDown().
File be/src/runtime/

PS4, Line 59: boost::
> remove

Line 66:       VLOG_QUERY << "new QueryState: query_id=" << query_id;
> Why is there no refcnt increment here?
it's a new one, and the refcnt is initialized to 1 (documented for the c'tor).

PS4, Line 69: boost::
> remove here and below.

PS4, Line 86: Thread::Run("query-exec-mgr",
            :       Substitute("exec-fragment-instance-$0", PrintId(instance_id)),
            :       &QueryExecMgr::StartFInstanceHelper, this, fis);
> Why is this not in the QueryState? It seems like it should be part of Regis
i felt both approaches are equally valid.

PS4, Line 95: StartFInstanceHelper
> Call this ExecFInstance() to make it clear that it blocks until instance co

PS4, Line 115: ReleaseQueryState(fis->query_state());
> You have a bug here - if a fragment instance finishes quickly (perhaps it d
i thought about this, but didn't see it as a bug.

why do you think it's a bug? the second fragment instance will simply re-create the qs.

PS4, Line 124: boost::lock_guard<mutex> l2(qs->refcnt_lock_);
             :   ++qs->refcnt_;
> Prefer atomics rather than heavyweight mutexes for the ref count. Fewer loc

PS4, Line 126: DCHECK_GT(qs->refcnt_, 0);
> What are you checking for here - overflow or that the refcnt_ didn't start 
semantically it's the same to check it here, and this requires less synchronization.

PS4, Line 147: // someone else might have gc'd the entry
> How? Only one caller to ReleaseQueryState() can set refcnt == 0.
after decreasing the refcnt to 0 in l138 someone else could have increased it again before
we get the lock in l145.
File be/src/runtime/query-exec-mgr.h:

PS4, Line 47: In both cases it brackets the instance execution with an increment/decrement
            :   /// of the refcount.
> This needs clarification - why is this relevant to callers? I think you mea
it expresses that the qs won't go away while the instance is executing.

PS4, Line 58: Otherwise returns nullptr
> AFAICT, GetQueryState() will return a QueryState even if its refcount == 0.
i'll change this comment to fit the existing behavior.

i like the existing behavior better, because it eliminates the possibility of someone trying
to register a qs because this function returned nullptr, despite the qs still being accessible.

PS4, Line 66: /// TODO: isn't this available in std:: now?
> remove, once we switch to std::mutex we'll get this one as well.

PS4, Line 72: QueryState (owned by us)
> Comment on how it's allocated.
you mean that it's calling new?

PS4, Line 75: /// Execute instance and clean up.
> Clean up what? Does this block until instance has finished?
File be/src/runtime/

PS4, Line 42: refcnt_(1)
> set this to 0. Callers should always manage the refcount with a balanced nu
File be/src/runtime/query-state.h:

PS4, Line 79: /// don't access query_ctx().desc_tbl, it's most likely for a different fragment
> I don't understand this comment. Is this going to change after batching? Ot
yes, this will change with the per-query exec rpc. it's broken right now because every fragment
instance shows up with its own descriptor table, and they're all slightly different.

PS4, Line 107: bool
> Does access to this need to be synchronised?
not so far.

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: I962ae6b7cb7dc0d07fbb8f70317aeb01d88d400b
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Marcel Kornacker <>
Gerrit-Reviewer: Alex Behm <>
Gerrit-Reviewer: Henry Robinson <>
Gerrit-Reviewer: Lars Volker <>
Gerrit-Reviewer: Marcel Kornacker <>
Gerrit-Reviewer: Matthew Jacobs <>
Gerrit-HasComments: Yes

View raw message