impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcel Kornacker (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-4623: Enable file handle cache
Date Sun, 21 May 2017 16:10:23 GMT
Marcel Kornacker has posted comments on this change.

Change subject: IMPALA-4623: Enable file handle cache

Patch Set 10:

File be/src/runtime/disk-io-mgr-handle-cache.h:

Line 88:   /// is set to true. Otherwise, the partition will try to construct a file handle
> Added a TODO for this. I plan on dealing with this when doing eviction by t
true, but you're describing a scenario with concurrent queries and updates. we don't make
any guarantees in that case. it would be fine for step 6 to notice that the mtime changed
when it tries to reopen that file, and return an error at that point.
File be/src/runtime/disk-io-mgr-handle-cache.inline.h:

Line 130:   // return code, and we close the file handle and remove it from the cache.
thanks for the explanation
File be/src/runtime/

Line 417:     } else {
> This is the number of bytes read by either hdfsPread or hdfsRead. In case o
so it's just num_bytes_read then. when i see 'last' or 'prev' or something like that in a
loop i usually expect this value gets set in iteration n and used in n+1.
File be/src/runtime/

Line 352:       int success = hdfsGetHedgedReadMetrics(fs_, &hedged_metrics);
you're getting metrics for the whole fs_, but every scan range does that on its own, and then
updates a global counter? how is that not double-counting?

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: Ibe5ff60971dd653c3b6a0e13928cfa9fc59d078d
Gerrit-PatchSet: 10
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Joe McDonnell <>
Gerrit-Reviewer: Dan Hecht <>
Gerrit-Reviewer: Joe McDonnell <>
Gerrit-Reviewer: Marcel Kornacker <>
Gerrit-Reviewer: Tim Armstrong <>
Gerrit-HasComments: Yes

View raw message