impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (JIRA)" <>
Subject [jira] [Resolved] (IMPALA-4703) Add reservation stress option for test coverage
Date Sun, 06 Aug 2017 19:56:00 GMT


Tim Armstrong resolved IMPALA-4703.
       Resolution: Fixed
    Fix Version/s: Impala 2.10.0

IMPALA-4703: reservation denial debug action

Add debug action to deny reservation increases with some probability.
This allows us to test various scenarios, particularly:
* The case when the node only gets its initial reservation and must
  run to completion without increasing its reservation.
* The case when there is some memory pressure and the node sometimes
  gets a reservation increase and sometimes doesn't.

E.g. to deny all reservation requests after an ExecNode has opened:


This was applied to test_spilling. It caught a bug in the PAGG
with spilling string aggregations.

This required some minor extensions to the debug actions.
* Allow debug actions that apply to all ExecNodes if node_id is -1.
* Allow passing parameters to debug actions. The current grammar of the
  actions is not well-oriented towards extension, so I resorted to using
  @ as a new delimiter.

I also optimised ExecDebugAction() so that it is much faster in the
common case and extended --disable_mem_pools to prevent the buffer pool
from holding onto unused buffers.

Change-Id: Ied39bb091b12156e5dc61b528c6c0cd8de3fe657
Reviewed-by: Tim Armstrong <>
Tested-by: Impala Public Jenkins

> Add reservation stress option for test coverage
> -----------------------------------------------
>                 Key: IMPALA-4703
>                 URL:
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Backend
>    Affects Versions: Impala 2.8.0
>            Reporter: Dan Hecht
>            Assignee: Tim Armstrong
>            Priority: Minor
>             Fix For: Impala 2.10.0
> We should add a stress option to allow us to exercise various reservation scenarios.
> # Initial reservation not granted -> reservation error status in Prepare().
> # All reservation requests beyond initial not granted -> all (valid) queries complete
> # Random % of reservation requests beyond initial not granted -> all (valid) queries
complete successfully.

This message was sent by Atlassian JIRA

View raw message