impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Impala Public Jenkins (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-3642: Adding backend addresses to error statuses for some scratch failures.
Date Tue, 12 Sep 2017 03:54:22 GMT
Impala Public Jenkins has submitted this change and it was merged.

Change subject: IMPALA-3642: Adding backend addresses to error statuses for some scratch failures.
......................................................................


IMPALA-3642: Adding backend addresses to error statuses for some scratch failures.

Adds GetBackendAddress() (which is host:port) to error messages stemming
from SCRATCH_LIMIT_EXCEEDED, SCRATCH_READ_TRUNCATED, and
SCRATCH_ALLOCATION_FAILED messages.

Testing:

* Unit tests assert the string is updated for SCRATCH_LIMIT_EXCEEDED
  and SCRATCH_ALLOCATION_FAILED. SCRATCH_READ_TRUNCATED doesn't
  have an existing test, and I didn't add a new one.

* Manually testing a query that spills after "chmod 000 /tmp/impala-scratch":
    $ chmod 000 /tmp/impala-scratch
    $ impala-shell
    [dev:21000] > set mem_limit=100m;
    MEM_LIMIT set to 100m
    [dev:21000] > select count(*) from tpch_parquet.lineitem join tpch_parquet.orders on
l_orderkey = o_orderkey;
    Query: select count(*) from tpch_parquet.lineitem join tpch_parquet.orders on l_orderkey
= o_orderkey
    Query submitted at: 2017-09-11 11:07:06 (Coordinator: http://dev:25000)
    Query progress can be monitored at: http://dev:25000/query_plan?query_id=5c48ff8f4103c194:1b40a6c00000000
    WARNINGS: Could not create files in any configured scratch directories (--scratch_dirs=/tmp/impala-scratch)
on backend 'dev:22002'. See logs for previous errors that may have prevented creating or writing
scratch files.
    Opening '/tmp/impala-scratch/5c48ff8f4103c194:1b40a6c00000000_08e8d63b-169d-4571-a0fe-c48fa08d73e6'
for write failed with errno=13 description=Error(13): Permission denied
    Opening '/tmp/impala-scratch/5c48ff8f4103c194:1b40a6c00000000_08e8d63b-169d-4571-a0fe-c48fa08d73e6'
for write failed with errno=13 description=Error(13): Permission denied
    Opening '/tmp/impala-scratch/5c48ff8f4103c194:1b40a6c00000000_08e8d63b-169d-4571-a0fe-c48fa08d73e6'
for write failed with errno=13 description=Error(13): Permission denied

Change-Id: If31a50fdf6031312d0348d48aeb8f9688274cac2
Reviewed-on: http://gerrit.cloudera.org:8080/7816
Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
Tested-by: Impala Public Jenkins
---
M be/src/runtime/bufferpool/buffer-pool-test.cc
M be/src/runtime/tmp-file-mgr-test.cc
M be/src/runtime/tmp-file-mgr.cc
M common/thrift/generate_error_codes.py
4 files changed, 13 insertions(+), 9 deletions(-)

Approvals:
  Impala Public Jenkins: Verified
  Tim Armstrong: Looks good to me, approved



-- 
To view, visit http://gerrit.cloudera.org:8080/7816
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If31a50fdf6031312d0348d48aeb8f9688274cac2
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Philip Zeyliger <philip@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Philip Zeyliger <philip@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message