impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew Jacobs (JIRA)" <>
Subject [jira] [Resolved] (IMPALA-5301) minicluster kudu needs mem limits set
Date Wed, 17 May 2017 23:52:04 GMT


Matthew Jacobs resolved IMPALA-5301.
       Resolution: Fixed
    Fix Version/s: Impala 2.9.0

commit d04f96b990c18ed17c82c0151978b56f936f0574
Author: Matthew Jacobs <>
Date:   Wed May 10 11:10:21 2017 -0700

    IMPALA-5301: Set Kudu minicluster memory limit
    By default, Kudu assumes it has 80% of system memory which
    is far too high for the minicluster. This sets a mem limit
    of 2gb and lowers the limit of the block cache. These values
    were tested on a gerrit-verify-dryrun job as well as an
    exhaustive run.
    This patch also simplifies TestKuduMemLimits which was
    unnecessarily creating a large table during test execution.
    Change-Id: I7fd7e1cd9dc781aaa672a2c68c845cb57ec885d5
    Reviewed-by: Todd Lipcon <>
    Reviewed-by: Tim Armstrong <>
    Tested-by: Impala Public Jenkins

> minicluster kudu needs mem limits set
> -------------------------------------
>                 Key: IMPALA-5301
>                 URL:
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Infrastructure
>    Affects Versions: Impala 2.8.0
>            Reporter: Matthew Jacobs
>              Labels: kudu
>             Fix For: Impala 2.9.0
> The minicluster runs Kudu without an explicit memory limit, and Kudu defaults to assuming
it can use 80% of system memory. We should set a limit to avoid overcommitting system memory.
> We need to keep the value low because aws test nodes ( are small, see
IMPALA-5297. I tested a limit of 1gb but that wasn't enough -- TestKuduMemLimits failed. We
may try something a bit larger, or perhaps find other ways to reduce the requirements of minicluster

This message was sent by Atlassian JIRA

View raw message