impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-4733: Avoid using several loading threads on one table.
Date Fri, 13 Jan 2017 16:28:28 GMT
Tim Armstrong has posted comments on this change.

Change subject: IMPALA-4733: Avoid using several loading threads on one table.
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/5707/1/fe/src/main/java/org/apache/impala/catalog/TableLoadingMgr.java
File fe/src/main/java/org/apache/impala/catalog/TableLoadingMgr.java:

Line 280:     tableLoadingSet_.remove(tblName);
There's still a window for a race here if multiple threads pull the same entry off the queue
at the same time. I think we can avoid it if we change the data structures slightly.

How about we change loadingTables_ to a map from TTableName to an atomic integer, and only
only remove tables from the map once the load finishes?

That way, the first loading thread to flip the atomic integer from 0 to 1 wins the race and
does the load. If another thread doesn't find the entry it knows that a load completed after
it was last added. If it finds the integer set to 1, it knows that a load is in progress.

Not ideal but it makes the race impossible and keeps this mechanism separate from loadingTables_,
which has its own lifecycle.


-- 
To view, visit http://gerrit.cloudera.org:8080/5707
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Idba5f1808e0b9cbbcf46245834d8ad38d01231cb
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message