impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joe McDonnell (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5386: Fix ReopenCachedHdfsFileHandle failure case
Date Tue, 30 May 2017 22:50:29 GMT
Hello Michael Ho,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/7020

to look at the new patch set (#2).

Change subject: IMPALA-5386: Fix ReopenCachedHdfsFileHandle failure case
......................................................................

IMPALA-5386: Fix ReopenCachedHdfsFileHandle failure case

This fixes two issues with the file handle cache.

The first issue is that ReopenCachedHdfsFileHandle can
destroy the passed in file handle without removing the
reference to it. The old file handle then refers to
a piece of memory that is not a handle in the cache,
so future use of the handle fails with an assert. The
fix is to always overwrite the reference to the file
handle when it has been destroyed.

The second issue is that query_test/test_hdfs_fd_caching.py
and custom_cluster/test_hdfs_fd_caching.py should not be
running on remote files (S3, ADLS, Isilon, remote
clusters). The file handle cache is not used for remote
files, so the expected semantics don't apply.

Change-Id: Iee982fa5e964f6c8969b2eb7e5f3eca89e793b3a
---
M be/src/runtime/disk-io-mgr-handle-cache.inline.h
M be/src/runtime/disk-io-mgr.cc
M tests/common/skip.py
M tests/custom_cluster/test_hdfs_fd_caching.py
M tests/query_test/test_hdfs_fd_caching.py
5 files changed, 13 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/20/7020/2
-- 
To view, visit http://gerrit.cloudera.org:8080/7020
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iee982fa5e964f6c8969b2eb7e5f3eca89e793b3a
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Joe McDonnell <joemcdonnell@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <joemcdonnell@cloudera.com>
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>

Mime
View raw message