impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew Jacobs (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5644: Reject queries if min reservation is too large
Date Wed, 16 Aug 2017 17:35:35 GMT
Matthew Jacobs has posted comments on this change.

Change subject: IMPALA-5644: Reject queries if min reservation is too large
......................................................................


Patch Set 1:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/7678/1/be/src/runtime/bufferpool/reservation-util.cc
File be/src/runtime/bufferpool/reservation-util.cc:

Line 23:   const double ReservationUtil::RESERVATION_MEM_FRACTION = 0.8;
> We don't normally indent inside namespaces.
Done


http://gerrit.cloudera.org:8080/#/c/7678/1/be/src/runtime/query-state.cc
File be/src/runtime/query-state.cc:

Line 153:   if (query_options().__isset.buffer_pool_limit
> The logic in admission control should also take into account buffer_pool_li
Good point. I'll update AC and remove the broken test case. I'll be sure to wait & rebase
my change on top of the change w/ your new test.


PS1, Line 155: max_reservation = query_options().buffer_pool_limit;
shouldn't max_reservation be the the minimum of query_options().buffer_pool_limit (if set)
and mem_limit (if set)?

This code implies that you can set a buffer_pool_limit > mem_limit and bypass the mem_limit
here. I'd assume the query would fail later.


http://gerrit.cloudera.org:8080/#/c/7678/1/be/src/scheduling/admission-controller.cc
File be/src/scheduling/admission-controller.cc:

Line 428:   // Checks related to pool max_mem_resources:
> Is there a reason behind the order of the checks? Might be helpful to docum
Not particularly, though I tried to think about what checks you'd want to be reported as failing
first since the user will only get an error with the first check to fail. I'll add a comment
at the top of this function.


PS1, Line 445: GetProcMemLimit
> Should we also be comparing the reservation against the process memory limi
I thought about it, though I think we should hold off for now. This check isn't great because
it's wrong if there are different process mem_limits across the cluster. Lets start planning
the next steps for admission control and see how we can fit it into the roadmap.


-- 
To view, visit http://gerrit.cloudera.org:8080/7678
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Iabe87ce8f460356cfe4d1be4d7092c5900f9d79b
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Matthew Jacobs <mj@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message