impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Impala Public Jenkins (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5377: Impala may crash if given a fragment instance while restarting
Date Fri, 02 Jun 2017 04:59:26 GMT
Impala Public Jenkins has submitted this change and it was merged.

Change subject: IMPALA-5377: Impala may crash if given a fragment instance while restarting
......................................................................


IMPALA-5377: Impala may crash if given a fragment instance while restarting

If an Impala daemon restarts quickly enough so that the statestore does
not detect a failure, coordinators will still issue fragment instances
to that daemon. There's a race in that situation, where the BE service
port was opened before ExecEnv::StartServices() had initialized the
process-wide memtracker; trying to run the finstance would then crash
the process.

This patch inverts the order of ExecEnv::StartServices() and starting
the backend and client servers.

Testing: reproduced by manually injecting sleeps after
be_server->Start() and running a query on the cluster. After the patch,
sleeping *before or after* be_server->Start() did not trigger a crash,
and the query failed as expected if it was executed in that narrow
window.

No automated testing added: this is a hard bug to hit without sleeps and
would need some repeated kill-and-restart loops to have a reasonable
chance of triggering.

Change-Id: I1d53b36304cb86c43e110e10cf76a39976ae3bd5
Reviewed-on: http://gerrit.cloudera.org:8080/7036
Reviewed-by: Dan Hecht <dhecht@cloudera.com>
Tested-by: Impala Public Jenkins
---
M be/src/service/impalad-main.cc
1 file changed, 10 insertions(+), 7 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Dan Hecht: Looks good to me, approved



-- 
To view, visit http://gerrit.cloudera.org:8080/7036
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I1d53b36304cb86c43e110e10cf76a39976ae3bd5
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <henry@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Henry Robinson <henry@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message