Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 75EA3200D0F for ; Fri, 29 Sep 2017 20:31:12 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 745F41609D1; Fri, 29 Sep 2017 18:31:12 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 93E4F1609BC for ; Fri, 29 Sep 2017 20:31:11 +0200 (CEST) Received: (qmail 25647 invoked by uid 500); 29 Sep 2017 18:31:10 -0000 Mailing-List: contact user-help@impala.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@impala.incubator.apache.org Delivered-To: mailing list user@impala.incubator.apache.org Received: (qmail 25637 invoked by uid 99); 29 Sep 2017 18:31:10 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 29 Sep 2017 18:31:10 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 4CE34CA2A0 for ; Fri, 29 Sep 2017 18:31:10 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.379 X-Spam-Level: ** X-Spam-Status: No, score=2.379 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=cloudera.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id N6BFiefW_GO7 for ; Fri, 29 Sep 2017 18:31:04 +0000 (UTC) Received: from mail-ua0-f170.google.com (mail-ua0-f170.google.com [209.85.217.170]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id A62105F39F for ; Fri, 29 Sep 2017 18:31:03 +0000 (UTC) Received: by mail-ua0-f170.google.com with SMTP id t36so268041uah.10 for ; Fri, 29 Sep 2017 11:31:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudera.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=f/ndElsUG92xuo7VWyNg4bvWt0RjqT7LNunz7hjrEbQ=; b=T3/OPCGT+J0WncMZ0ekGHlPsbtOWUgfZuf4v0ibmjPvjaN+pyAM0y75EKaYNxr6bUn bYzQypaVq0hywBFEBVRYkuiNplKGCoSHclf8WQjntVC1fVBsOf2yM+KQp5Rdpuf4cOS2 genOs4KIxSb+zUrNS5VG+n8iFjKilLFgx0dedtABdU5+llLG1KaUYDbJ142eUfuDIrNy CwWqKy0LA48iJux0NN5y6raahNWHLsbIENCmvbYjoQJqvrMOM0k5CoHa6qmscjMNBSfT uxgQ4EQqtRif1f4Bxgocbkb4WnRJoniHcKD2fhBM7y3KFxdT7ICffPdX0jNo1iKaWo1F j9QQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=f/ndElsUG92xuo7VWyNg4bvWt0RjqT7LNunz7hjrEbQ=; b=SphwQGcCL9WUjFaPZProTiMYr7oC6oXdNKSmNUQxBJq5KNEA1nMvRgKUha0wG2RWnb 6yiOXWYEoklYmeolg9UPqtRC6Oew8zSrWjqwJWfGB7NOnYsWPeaD/rHj+aKUHPz6e/pW jRwhBwdurtYmu4DjzXez01SOQS6whxU8WmhKo67XPsnE821llAF7DfK5u6QQIGKpse7H i3CLb1Z+oRbI1piyLKHtV8kY8oIa0hgtv84RNUwZtqDILsLHCnlQsERjn7gvMdvwCRFQ HkMadi29QG1Elth6yupoLS7TxGHSpg89kogcRCJPjjmap1eWV3J5Qj2o8hKCuueTWmlM 30Gw== X-Gm-Message-State: AHPjjUi9J/pQLVwXYXMWAOl+ukESaazf4zNpr3IJs07F2lpF1d80GBUd PT1n+GwxXBkbFRQ4ynp/CTEDwWG1SWa3SYoL/OeWwbtQ X-Google-Smtp-Source: AOwi7QChJ5XpDSVFYXEeKTIP+pQ7Oeymoehx7sNA3Q+tzw91/pO7Bz5mGP6EKIOCwuDbtbbaZjRJ/IEf6vhElK4fqKE= X-Received: by 10.176.20.169 with SMTP id d38mr5342758uae.184.1506709861867; Fri, 29 Sep 2017 11:31:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.174.72 with HTTP; Fri, 29 Sep 2017 11:30:31 -0700 (PDT) In-Reply-To: References: From: Tim Armstrong Date: Fri, 29 Sep 2017 11:30:31 -0700 Message-ID: Subject: Re: Impala Memory Leak (version 2.7.0-cdh5.10.2 ) To: user@impala.incubator.apache.org Content-Type: multipart/alternative; boundary="001a1145448618c36f055a583a04" archived-at: Fri, 29 Sep 2017 18:31:12 -0000 --001a1145448618c36f055a583a04 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable It's not a memory leak, it's a query lifecycle issue. You have fragments from two queries 2741ad08c43062bc:608bdb9500000000 and 8041bf903a5c7916:2eba50b100000000 that haven't been torn down. There are various ways that this could happen but I don't know based on the information provided. I would check if those queries are still running and if you can cancel them via Cloudera Manager or the Impala daemon web UI on the coordinator. On Fri, Sep 29, 2017 at 1:11 AM, Dong Bo =E8=91=A3=E5=8D=9A wrote: > Hi Experts, > > > > We are running queries on impalad version 2.7.0-cdh5.10.2 RELEASE (build = 38c989c0330ea952133111e41965ff9af96412d3), and issued with a memory leak pr= oblem on the impala deamon . > > Once query end with error Memory limit exceeded , the allocated memory > won=E2=80=99t be free until impala deamon restarted. Memory limit excee= ded is in > attachement. > > > > I find jira tickets https://issues.apache.org/jira/browse/IMPALA-3589 , > https://issues.apache.org/jira/browse/IMPALA-3582 with same issue as I am > facing tagged as =E2=80=9CFixed=E2=80=9D in version 2.6.0, but strange I = am still facing > the issue in a higher version. > > > > Please kindly advise , it will be very helpful. > > > > Thanks & Regards > > Carl Dong > > > > > > > --001a1145448618c36f055a583a04 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
It's not a memory leak, it's a query lifecycl= e issue. You have fragments from two queries 2741ad08c43062bc:608bdb9500000= 000 and 8041bf903a5c7916:2eba50b100000000 that haven't been torn down. = There are various ways that this could happen but I don't know based on= the information provided.

I would check if those queries are = still running and if you can cancel them via Cloudera Manager or the Impala= daemon web UI on the coordinator.

=
On Fri, Sep 29, 2017 at 1:11 AM, Dong Bo =E8=91= =A3=E5=8D=9A <dongbo@fosun.com> wrote:

Hi E= xperts,

=C2=A0
We are running queries on impalad version 2.7.0-cdh5.10.2 R=
ELEASE (build 38c989c0330ea952133111e41965ff9af96412d3), and issued wi=
th a memory leak problem on the impala deamon . =C2=A0=C2=A0=C2=A0=C2=A0 <=
/pre>

Once=
 query end with error Memory limit exceeded , the allocated memory won=E2=
=80=99t be free until impala deamon restarted. =C2=A0=C2=A0Memory limit exc=
eeded is in attachement.


<=
/u>=C2=A0


I fi=
nd jira tickets
https://issues.apache.org/jira/browse/IMPALA-3589 ,
https://issues.apache.org/jira/browse/IMPALA-3582 with same i=
ssue as I am facing tagged as =E2=80=9CFixed=E2=80=9D in version 2.6.0, but=
 strange I am still facing the issue in a higher version.


<=
/u>=C2=A0


Plea=
se kindly advise , it will be very helpful.


<=
/u>=C2=A0


Than=
ks & Regards


Carl=
 Dong


=C2=A0


=C2=A0


=C2=A0

--001a1145448618c36f055a583a04--