impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Henry Robinson (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5056: Ensure analysis uses 'fresh' catalog after metadata loading
Date Mon, 05 Jun 2017 00:30:56 GMT
Hello Dimitris Tsirogiannis,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/7045

to look at the new patch set (#4).

Change subject: IMPALA-5056: Ensure analysis uses 'fresh' catalog after metadata loading
......................................................................

IMPALA-5056: Ensure analysis uses 'fresh' catalog after metadata loading

If a query was waiting for metadata to be loaded at the same time as the
statestore restarted, it would not correctly see the metadata even after
the statestore had recovered and sent it.

This is because the AnalysisContext used for analysis is created with a
reference to a ImpaladCatalog once, before the analysis->metadata-load
loop started. If the catalog sent a *complete* update (which it would do
after a statestore restart), the ImpaladCatalog would be replaced, not
updated, and so the tables would exist in the new object, but not the
old one that the AnalysisContext had a reference to.

The fix is to update the AnalysisContext's catalog every time through
the loop, taking a recent reference to the ImpaladCatalog every
time. That way, if the ImpaladCatalog is updated, the next time through
the loop the analysis phase will see those changes.

Also fix a potential bug where access to ImpaladCatalog is not
synchronized between update and analysis threads, by using an
AtomicReference<ImpaladCatalog>.

Testing: manual - queries completed after statestore restarts with this
patch, where previously they consumed 100% CPU spinning forever.

Change-Id: I574d69fa75198499523dc291fbbd0d7e3d8d968f
---
M fe/src/main/java/org/apache/impala/analysis/AnalysisContext.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
2 files changed, 35 insertions(+), 26 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/7045/4
-- 
To view, visit http://gerrit.cloudera.org:8080/7045
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I574d69fa75198499523dc291fbbd0d7e3d8d968f
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <henry@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
Gerrit-Reviewer: Henry Robinson <henry@cloudera.com>

Mime
View raw message