impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Impala Public Jenkins (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-4765: Avoid using several loading threads on one table.
Date Sun, 15 Jan 2017 08:38:14 GMT
Impala Public Jenkins has submitted this change and it was merged.

Change subject: IMPALA-4765: Avoid using several loading threads on one table.
......................................................................


IMPALA-4765: Avoid using several loading threads on one table.

When there are multiple concurrent requests to the catalogd to
prioritize loading the same table, then several catalog loading
threads may end up waiting for that single table to be loaded,
effectively reducing the number of catalog loading threads. In
extreme examples, this might degrade to serial loading of tables.

This patch augments the existing data structures and code to
prevent using several loading threads for the same table.
Some of the existing data structures and code could be
consolidated/simplified but this patch does not try to address
that issue to minimize the risk of this change.

Testing: I could easily reproduce the bug locally with the steps
described in the JIRA. After this patch, I could not observe threads
being wasted anymore.

Change-Id: Idba5f1808e0b9cbbcf46245834d8ad38d01231cb
Reviewed-on: http://gerrit.cloudera.org:8080/5707
Reviewed-by: Alex Behm <alex.behm@cloudera.com>
Tested-by: Impala Public Jenkins
---
M fe/src/main/java/org/apache/impala/catalog/TableLoadingMgr.java
1 file changed, 34 insertions(+), 21 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Alex Behm: Looks good to me, approved



-- 
To view, visit http://gerrit.cloudera.org:8080/5707
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Idba5f1808e0b9cbbcf46245834d8ad38d01231cb
Gerrit-PatchSet: 7
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bharathv@cloudera.com>
Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Jim Apple <jbapple-impala@apache.org>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message