impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (JIRA)" <j...@apache.org>
Subject [jira] [Created] (IMPALA-5488) TestValidateMetrics fails sometimes when running tests on Isilon
Date Mon, 12 Jun 2017 19:44:01 GMT
Tim Armstrong created IMPALA-5488:
-------------------------------------

             Summary: TestValidateMetrics fails sometimes when running tests on Isilon
                 Key: IMPALA-5488
                 URL: https://issues.apache.org/jira/browse/IMPALA-5488
             Project: IMPALA
          Issue Type: Bug
          Components: Backend
    Affects Versions: Impala 2.10.0
            Reporter: Tim Armstrong
            Assignee: Joe McDonnell
            Priority: Blocker


This happened on commit e0f69ca1da448246a6260099c64f4cc5862188a2. I'm unsure when it started
failing, because it was masked by IMPALA-5423. The last green Isilon build was 014c5603f867907963f3821948f90d526e9a4789,
which is before the file handle cache change was merged.

{code}
23:32:26 ============ 1367 passed, 82 skipped, 28 xfailed in 3965.31 seconds ============
23:32:26 ============================= test session starts ==============================
23:32:26 platform linux2 -- Python 2.6.6, pytest-2.9.2, py-1.4.32, pluggy-0.3.1 -- /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/repos/Impala/bin/../infra/python/env/bin/python
23:32:26 cachedir: .cache
23:32:26 rootdir: /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/repos/Impala/tests,
inifile: pytest.ini
23:32:26 plugins: random-0.2, xdist-1.15.0
23:32:26 collecting ... collected 2 items
23:32:26 
23:32:26 verifiers/test_verify_metrics.py::TestValidateMetrics::test_metrics_are_zero FAILED
23:32:26 verifiers/test_verify_metrics.py::TestValidateMetrics::test_num_unused_buffers PASSED
23:32:26 
23:32:26  generated xml file: /data/jenkins/workspace/impala-umbrella-build-and-test-isilon/repos/Impala/logs/ee_tests/results/TEST-impala-verify-metrics.xml

23:32:26 =========================== short test summary info ============================
23:32:26 FAIL verifiers/test_verify_metrics.py::TestValidateMetrics::()::test_metrics_are_zero
23:32:26 =================================== FAILURES ===================================
23:32:26 __________________ TestValidateMetrics.test_metrics_are_zero ___________________
23:32:26 verifiers/test_verify_metrics.py:34: in test_metrics_are_zero
23:32:26     verifier.verify_metrics_are_zero()
23:32:26 verifiers/metric_verifier.py:44: in verify_metrics_are_zero
23:32:26     self.wait_for_metric(metric, 0, timeout)
23:32:26 verifiers/metric_verifier.py:59: in wait_for_metric
23:32:26     self.impalad_service.wait_for_metric_value(metric_name, expected_value, timeout)
23:32:26 common/impala_service.py:89: in wait_for_metric_value
23:32:26     (metric_name, expected_value, timeout)
23:32:26 E   AssertionError: Metric value impala-server.io-mgr.num-open-files did not reach
value 0 in 60s
23:32:26 ----------------------------- Captured stderr call -----------------------------
23:32:26 MainThread: Getting metric: impala-server.hash-table.total-bytes from localhost:25000
23:32:26 MainThread: Metric 'impala-server.hash-table.total-bytes' has reach desired value:
0
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
23:32:26 MainThread: Getting metric: impala-server.io-mgr.num-open-files from localhost:25000
23:32:26 MainThread: Waiting for metric value 'impala-server.io-mgr.num-open-files'=0. Current
value: 2
23:32:26 MainThread: Sleeping 1s before next retry.
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message