impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-4835: Part 1: simplify I/O mgr mem mgmt and cancellation
Date Fri, 17 Nov 2017 17:44:13 GMT
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8414 )

Change subject: IMPALA-4835: Part 1: simplify I/O mgr mem mgmt and cancellation
......................................................................


Patch Set 8:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/8414/8/be/src/runtime/io/request-context.h
File be/src/runtime/io/request-context.h:

http://gerrit.cloudera.org:8080/#/c/8414/8/be/src/runtime/io/request-context.h@171
PS8, Line 171:     // boost doesn't let us dcheck that the reader lock is taken
> Outdated comment
Done


http://gerrit.cloudera.org:8080/#/c/8414/8/be/src/runtime/io/request-context.h@294
PS8, Line 294:   InternalQueue<ScanRange> cached_ranges_;
> It seems these queues don't need the spinlocks in them. But we don't need t
Good point, we can fix that now - it makes it confusing about whether holding 'lock_' is actually
required. We have an InternalList class that is the same thing without the locking.


http://gerrit.cloudera.org:8080/#/c/8414/8/be/src/runtime/io/request-context.h@366
PS8, Line 366:     void DecrementRequestThread(const boost::unique_lock<boost::mutex>&
context_lock,
> How about DecrementDiskThread? I don't see the term "request thread" elsewh
Done


http://gerrit.cloudera.org:8080/#/c/8414/8/be/src/runtime/io/request-context.h@378
PS8, Line 378:       if (is_on_queue_.Load() == 0 && num_threads_in_op_.Load() ==
0 && !done_) {
> I see the correctness here. It's pretty subtle.
Yeah it took me a while to figure it out. I think we could simplify it further but I stopped
at documenting why it was correct. I think this patch is already large.


http://gerrit.cloudera.org:8080/#/c/8414/8/be/src/runtime/io/request-context.h@386
PS8, Line 386:     /// If done is true, it means that this request must not be on this disk's
queue
> Is this statement true? I think RequestContext::Cancel() can enqueue the co
I'm pretty sure it's true. Validate() does check this. Cancel() doesn't actually set done
or even wait for the disks to be done - instead it relies on the disk threads to see that
the context is cancelled and set "done". I tried to improve the comment in Cancel() a little
bit.



-- 
To view, visit http://gerrit.cloudera.org:8080/8414
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: If5cb42437d11c13bc4a55c3ab426b66777332bd1
Gerrit-Change-Number: 8414
Gerrit-PatchSet: 8
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Tianyi Wang <twang@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Comment-Date: Fri, 17 Nov 2017 17:44:13 +0000
Gerrit-HasComments: Yes

Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message