Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id F2A96200D27 for ; Wed, 25 Oct 2017 19:12:28 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id F100B160BDA; Wed, 25 Oct 2017 17:12:28 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 429AA1609CE for ; Wed, 25 Oct 2017 19:12:28 +0200 (CEST) Received: (qmail 64290 invoked by uid 500); 25 Oct 2017 17:12:27 -0000 Mailing-List: contact dev-help@impala.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@impala.incubator.apache.org Delivered-To: mailing list dev@impala.incubator.apache.org Received: (qmail 64273 invoked by uid 99); 25 Oct 2017 17:12:27 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 25 Oct 2017 17:12:27 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 5A4EDC10BB for ; Wed, 25 Oct 2017 17:12:26 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.379 X-Spam-Level: ** X-Spam-Status: No, score=2.379 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=cloudera.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id dcgHMuISWReE for ; Wed, 25 Oct 2017 17:12:25 +0000 (UTC) Received: from mail-it0-f48.google.com (mail-it0-f48.google.com [209.85.214.48]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id EC4D85FB9F for ; Wed, 25 Oct 2017 17:12:24 +0000 (UTC) Received: by mail-it0-f48.google.com with SMTP id r127so1861196itb.5 for ; Wed, 25 Oct 2017 10:12:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudera.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=PTTEOK7dM9Go5sKxJtb0uLZldp6JqLXJdzEGb8ghMVA=; b=FUs/jsvcp6pJiWDMD1ocoHIMvRlQelkv2RB5dHxai5WWwH2rN/Nw1LjWusmHvDFv8S oTlQWqIKpDJYltUxJURXirK/HuR+LNlifGFrCmlPwhFgwkXl3GdTVKkOXJFGC6gUjW9f 6I0BQKVzxlWkPsUG8pM1LwrA5oYYvZx/hePsPDpSBee3NYWp76H4dQdQW32hfWkf6jF6 zsm8w5c/FZc4zMiE9bae0KsN5etnRtVrbqe/2eRhsrB1CH2m3/k/gAssUyNPEQvOJRDP ClXE4msf0LnFyayXSVT2WPKye2C3UcY4mlMugt4ChnH1M2t7CiCI8wqC6AfAd1KljS6b 0SIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=PTTEOK7dM9Go5sKxJtb0uLZldp6JqLXJdzEGb8ghMVA=; b=hzSuvBBvx06TC+NtcbGdThl6n79oxkZ1eL+5swFCpiyyRvxw2zCrU0k6SEQwLEq2Oo 0obaMByYIImwPNuXZXhK9OK1Y/FK6J9IEWFzDCcfRY9b4ZYUlTuh5OJYm5PbSy7vP6pJ QlMO/PSpzbIIvu7BqzT6pAXNqDgGoJrMw9GYWiiTiXCJHKGPzp9+C9nefU3Qt0qgnH9B NoxwWDHUJzEDmzCK9Tx6HJoH2DsteLinjVcaPcBhdIdlAPHWoTLpj2GUAOl9RqnPUJA4 oqv4qTJDudTuZAc4Fub3zWB26FSWoIKeeuvDbnpC/Sgn1F55SNP6t+OFNURd4GEllkLh Pteg== X-Gm-Message-State: AMCzsaUriYEimE8Bd/4ygsT0vvGsmVJtY4ahREyji/wMznOYbljPLVn8 CCWOjpT5Envsr/qAgw6S1ULci6FddbDytuU+ifYym6co X-Google-Smtp-Source: ABhQp+QaGWSUsulnzDOebbruHCf2TRAvNCEQ0Xkof9mJKPSiwH4Kp8p8miVxcmnLxgGqLDjQs77/uZrSxaofBVsiYcU= X-Received: by 10.36.242.132 with SMTP id j126mr2902972ith.141.1508951537926; Wed, 25 Oct 2017 10:12:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.13.136 with HTTP; Wed, 25 Oct 2017 10:11:36 -0700 (PDT) In-Reply-To: References: From: Mostafa Mokhtar Date: Wed, 25 Oct 2017 10:11:36 -0700 Message-ID: Subject: Re: Does this mem-tracker.h assertion ring a bell? To: dev@impala.incubator.apache.org Content-Type: multipart/alternative; boundary="94eb2c1231d866e285055c6228d0" archived-at: Wed, 25 Oct 2017 17:12:29 -0000 --94eb2c1231d866e285055c6228d0 Content-Type: text/plain; charset="UTF-8" Maybe related to https://issues.apache.org/jira/browse/IMPALA-6099? On Wed, Oct 25, 2017 at 10:02 AM, Philip Zeyliger wrote: > Hi folks, > > I'm debugging some test failures related to an LLVM/AvroCodegen patch I've > got going on. The failures are in the parallel EE tests, and most of them > are complaining that Impala is out to lunch. It looks like the following > assertion is firing, causing an impalad to fail, causing many tests to > start failing. (I've also got a minidump, but the build was on > jenkins.impala.io, so I don't think I have the symbols/binaries to use > it.) > > If this sort of thing rings a bell for anyone, please holler! > > Obviously I'll work on reproducing this locally to figure it out. > > F1025 02:20:43.786911 82485 mem-tracker.h:231] Check failed: > tracker->consumption_->current_value() >= 0 (-1052615 vs. 0) > Runtime Filter (Coordinator): Total=-1.00 MB Peak=1.00 MB > *** Check failure stack trace: *** > @ 0x2f1e11d google::LogMessage::Fail() > @ 0x2f1f9c2 google::LogMessage::SendToLog() > @ 0x2f1daf7 google::LogMessage::Flush() > @ 0x2f210be google::LogMessageFatal::~LogMessageFatal() > @ 0x17425fb impala::MemTracker::Release() > @ 0x1fa7e8b impala::Coordinator::UpdateFilter() > @ 0x186e3cf impala::ImpalaServer::UpdateFilter() > @ 0x18d824f impala::ImpalaInternalService::UpdateFilter() > @ 0x1dda35a > impala::ImpalaInternalServiceProcessor::process_UpdateFilter() > @ 0x1dd8308 > impala::ImpalaInternalServiceProcessor::dispatchCall() > @ 0x15410ea apache::thrift::TDispatchProcessor::process() > @ 0x171042b > apache::thrift::server::TAcceptQueueServer::Task::run() > @ 0x170c307 impala::ThriftThread::RunRunnable() > @ 0x170da13 boost::_mfi::mf2<>::operator()() > @ 0x170d8a9 boost::_bi::list3<>::operator()<>() > @ 0x170d5f5 boost::_bi::bind_t<>::operator()() > @ 0x170d508 > boost::detail::function::void_function_obj_invoker0<>::invoke() > @ 0x171bdfc boost::function0<>::operator()() > @ 0x19f3393 impala::Thread::SuperviseThread() > @ 0x19fbf26 boost::_bi::list4<>::operator()<>() > @ 0x19fbe69 boost::_bi::bind_t<>::operator()() > @ 0x19fbe2c boost::detail::thread_data<>::run() > @ 0x20a7c9a thread_proxy > @ 0x7fe6536186ba start_thread > @ 0x7fe65334e3dd clone > r.java:81) > --94eb2c1231d866e285055c6228d0--