Mailing-List: contact issues-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@impala.incubator.apache.org
Date: Wed, 17 May 2017 23:52:04 +0000 (UTC)
From: "Matthew Jacobs (JIRA)" <jira@apache.org>
To: issues@impala.incubator.apache.org
Message-ID: <JIRA.13071015.1494465958000.235859.1495065124334@Atlassian.JIRA>
In-Reply-To: <JIRA.13071015.1494465958000@Atlassian.JIRA>
References: <JIRA.13071015.1494465958000@Atlassian.JIRA> <JIRA.13071015.1494465958723@jira-lw-us.apache.org>
Subject: [jira] [Resolved] (IMPALA-5301) minicluster kudu needs mem limits
 set
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 17 May 2017 23:52:09 -0000


     [ https://issues.apache.org/jira/browse/IMPALA-5301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthew Jacobs resolved IMPALA-5301.
------------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.9.0

commit d04f96b990c18ed17c82c0151978b56f936f0574
Author: Matthew Jacobs <mj@cloudera.com>
Date:   Wed May 10 11:10:21 2017 -0700

    IMPALA-5301: Set Kudu minicluster memory limit
    
    By default, Kudu assumes it has 80% of system memory which
    is far too high for the minicluster. This sets a mem limit
    of 2gb and lowers the limit of the block cache. These values
    were tested on a gerrit-verify-dryrun job as well as an
    exhaustive run.
    
    This patch also simplifies TestKuduMemLimits which was
    unnecessarily creating a large table during test execution.
    
    Change-Id: I7fd7e1cd9dc781aaa672a2c68c845cb57ec885d5
    Reviewed-on: http://gerrit.cloudera.org:8080/6844
    Reviewed-by: Todd Lipcon <todd@apache.org>
    Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
    Tested-by: Impala Public Jenkins


> minicluster kudu needs mem limits set
> -------------------------------------
>
>                 Key: IMPALA-5301
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5301
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Infrastructure
>    Affects Versions: Impala 2.8.0
>            Reporter: Matthew Jacobs
>              Labels: kudu
>             Fix For: Impala 2.9.0
>
>
> The minicluster runs Kudu without an explicit memory limit, and Kudu defaults to assuming it can use 80% of system memory. We should set a limit to avoid overcommitting system memory.
> We need to keep the value low because aws test nodes (jenkins.impala.io) are small, see IMPALA-5297. I tested a limit of 1gb but that wasn't enough -- TestKuduMemLimits failed. We may try something a bit larger, or perhaps find other ways to reduce the requirements of minicluster tests.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)