Return-Path: X-Original-To: apmail-nifi-users-archive@minotaur.apache.org Delivered-To: apmail-nifi-users-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 81044183D4 for ; Fri, 13 Nov 2015 05:29:06 +0000 (UTC) Received: (qmail 12792 invoked by uid 500); 13 Nov 2015 05:29:06 -0000 Delivered-To: apmail-nifi-users-archive@nifi.apache.org Received: (qmail 12763 invoked by uid 500); 13 Nov 2015 05:29:06 -0000 Mailing-List: contact users-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@nifi.apache.org Delivered-To: mailing list users@nifi.apache.org Received: (qmail 12753 invoked by uid 99); 13 Nov 2015 05:29:06 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 13 Nov 2015 05:29:06 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id ED109180E67 for ; Fri, 13 Nov 2015 05:29:05 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.12 X-Spam-Level: X-Spam-Status: No, score=-0.12 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id zjxXu4PGytjn for ; Fri, 13 Nov 2015 05:29:02 +0000 (UTC) Received: from mail-ig0-f174.google.com (mail-ig0-f174.google.com [209.85.213.174]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with ESMTPS id 2E41420CB8 for ; Fri, 13 Nov 2015 05:29:01 +0000 (UTC) Received: by igcph11 with SMTP id ph11so9107761igc.1 for ; Thu, 12 Nov 2015 21:28:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=bNvcy/qtDQf48TjcHxij+4gB5sKa5nJwdcEIK8VgofM=; b=HWoroRgxRQ1QjhE0MiTgtrddp5UUGtfkVxPmKlfLmDHqqg0QOdYKgCL6ANXaqt30JB XsHXGNGHwLjHPLBc3qIJvRuVgwvWCYFR7FfmohDeuZ9WVaL9f7c9QN6vqaEXYxh3EBsj NSEDCAdi0VhMIOnoag+W0WD6UbQGpy7QRuCK7pbKqtdBYFvStnq7PP5PKYs3PYN4+34e f3JOl5o2kfrfHX8Al5nh/4VtUmwNIAhd2UwWy6GaOSpZPoht5D5fSW9scUKAjajFmWGb mRMnytPCY+zvwRazZAu2W5n6rPGJJH/2gZti1FHTj6/V9SKylr+9BJ8pcFAnsVS6HPec ViVQ== MIME-Version: 1.0 X-Received: by 10.50.43.198 with SMTP id y6mr1338383igl.63.1447392534422; Thu, 12 Nov 2015 21:28:54 -0800 (PST) Received: by 10.36.219.65 with HTTP; Thu, 12 Nov 2015 21:28:54 -0800 (PST) In-Reply-To: References: <564574A4.2000504@gmail.com> Date: Fri, 13 Nov 2015 00:28:54 -0500 Message-ID: Subject: Re: High CPU usage in FileSystemRepository.java From: Joe Witt To: users@nifi.apache.org Content-Type: text/plain; charset=UTF-8 sorry - i see now :-) Thanks for the analysis. Will dig in. Joe On Fri, Nov 13, 2015 at 12:28 AM, Joe Witt wrote: > Adam, > > Are you on a recent master build? > > Thanks > Joe > > On Fri, Nov 13, 2015 at 12:27 AM, Adam Lamar wrote: >> Hi everybody! >> >> I'm following up from my previous thread about high CPU usage in GetSQS. I >> ran into high CPU usage while developing a patch for that processor, and >> while investigating with "top", I noticed one NiFi thread in particular >> showed high CPU usage, even after turning off all processors and restarting >> NiFi. >> >> A jstack showed this thread was busy at FileSystemRepository.java line 1287 >> [1]. Since that is a continue statement, it suggests that the thread was >> churning in the surrounding for loop. >> >> I didn't debug any further, but I did add a sleep statement just before the >> continue, and CPU usage dropped wildly, settling around 2-4%. >> >> I hope this is useful information, and I'm happy to debug further if needed. >> >> Cheers, >> Adam >> >> [1] >> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/repository/FileSystemRepository.java#L1287