impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philip Zeyliger (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-3642: Adding backend addresses to error statuses for some scratch failures.
Date Mon, 11 Sep 2017 18:13:44 GMT
Philip Zeyliger has uploaded a new patch set (#2).

Change subject: IMPALA-3642: Adding backend addresses to error statuses for some scratch failures.
......................................................................

IMPALA-3642: Adding backend addresses to error statuses for some scratch failures.

Adds GetBackendAddress() (which is host:port) to error messages stemming
from SCRATCH_LIMIT_EXCEEDED, SCRATCH_READ_TRUNCATED, and
SCRATCH_ALLOCATION_FAILED messages.

Testing:

* Unit tests assert the string is updated for SCRATCH_LIMIT_EXCEEDED
  and SCRATCH_ALLOCATION_FAILED. SCRATCH_READ_TRUNCATED doesn't
  have an existing test, and I didn't add a new one.

* Manually testing a query that spills after "chmod 000 /tmp/impala-scratch":
    $ chmod 000 /tmp/impala-scratch
    $ impala-shell
    [dev:21000] > set mem_limit=100m;
    MEM_LIMIT set to 100m
    [dev:21000] > select count(*) from tpch_parquet.lineitem join tpch_parquet.orders on
l_orderkey = o_orderkey;
    Query: select count(*) from tpch_parquet.lineitem join tpch_parquet.orders on l_orderkey
= o_orderkey
    Query submitted at: 2017-09-11 11:07:06 (Coordinator: http://dev:25000)
    Query progress can be monitored at: http://dev:25000/query_plan?query_id=5c48ff8f4103c194:1b40a6c00000000
    WARNINGS: Could not create files in any configured scratch directories (--scratch_dirs=/tmp/impala-scratch)
on backend 'dev:22002'. See logs for previous errors that may have prevented creating or writing
scratch files.
    Opening '/tmp/impala-scratch/5c48ff8f4103c194:1b40a6c00000000_08e8d63b-169d-4571-a0fe-c48fa08d73e6'
for write failed with errno=13 description=Error(13): Permission denied
    Opening '/tmp/impala-scratch/5c48ff8f4103c194:1b40a6c00000000_08e8d63b-169d-4571-a0fe-c48fa08d73e6'
for write failed with errno=13 description=Error(13): Permission denied
    Opening '/tmp/impala-scratch/5c48ff8f4103c194:1b40a6c00000000_08e8d63b-169d-4571-a0fe-c48fa08d73e6'
for write failed with errno=13 description=Error(13): Permission denied

Change-Id: If31a50fdf6031312d0348d48aeb8f9688274cac2
---
M be/src/runtime/bufferpool/buffer-pool-test.cc
M be/src/runtime/tmp-file-mgr-test.cc
M be/src/runtime/tmp-file-mgr.cc
M common/thrift/generate_error_codes.py
4 files changed, 13 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/16/7816/2
-- 
To view, visit http://gerrit.cloudera.org:8080/7816
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If31a50fdf6031312d0348d48aeb8f9688274cac2
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Philip Zeyliger <philip@cloudera.com>
Gerrit-Reviewer: Philip Zeyliger <philip@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message