From issues-return-94301-archive-asf-public=cust-asf.ponee.io@nifi.apache.org Tue Mar 17 20:23:03 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id AFC4918066D for ; Tue, 17 Mar 2020 21:23:02 +0100 (CET) Received: (qmail 27906 invoked by uid 500); 17 Mar 2020 20:23:02 -0000 Mailing-List: contact issues-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@nifi.apache.org Delivered-To: mailing list issues@nifi.apache.org Received: (qmail 27897 invoked by uid 99); 17 Mar 2020 20:23:02 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Mar 2020 20:23:02 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 50BDBE2E12 for ; Tue, 17 Mar 2020 20:23:01 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 733BF780402 for ; Tue, 17 Mar 2020 20:23:00 +0000 (UTC) Date: Tue, 17 Mar 2020 20:23:00 +0000 (UTC) From: "Joe Witt (Jira)" To: issues@nifi.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (NIFI-7266) NIFI 1.4.0 gets unresponsive after heavy load MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/NIFI-7266?page=3Dcom.atlassian.= jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D17061= 167#comment-17061167 ]=20 Joe Witt commented on NIFI-7266: -------------------------------- These are extremely old versions of NiFi here and very limited information = on the flow/configuration details. Hard to respond meaningfully. Seeing a= sudden drop in performance could be related to iops throttling. Could be = related to heap being close to full (which really depends on flow specifics= as to whether that should happen). Or other items. Would require a lot mo= re details/logs/etc.. to have a chance to be helpful. > NIFI 1.4.0 gets unresponsive after heavy load > --------------------------------------------- > > Key: NIFI-7266 > URL: https://issues.apache.org/jira/browse/NIFI-7266 > Project: Apache NiFi > Issue Type: Bug > Components: Configuration > Affects Versions: 1.2.0, 1.3.0, 1.4.0 > Reporter: Manuel Loayza > Priority: Trivial > Attachments: Screen Shot 2020-03-17 at 3.18.27 PM.png > > > We have 2 clusters (6 instances each one) running with NIFI 1.1.2 + JDK 8= u121 + Linux CentOS > The traffic get divided between those 2 clusters: > 1. TPS: 2700 - EAST cluster > 2. TPS: 980. - WEST cluster > We have tried to migrate to NIFI 1.2.0, 1.3.0, and 1.4.0, but the cluster= with higher TPS (EAST) got stuck after 4 hours of intensive traffic. Also = it web console got unresponsive. > I've tried many things to fix this thing, but only thing I got was to inc= rease the time from 4 to 6 hours before it fails > Our current instances are running on AWS and each EC2 instances has 8 cpu= s (c5.2xlarge), and 16GB RAM. > I've tried to use=C2=A0=C2=A0c5.4xlarge (it doubles the cpu and ram), but= I got the same outcome. > I don't have a clue to figure it out what the issue is.=C2=A0 Also I have= a datadog dashboard to track some java head metrics but everything looks n= ormal. > What should I do to find why those new better instances are failing? is i= t memory or disk space or threads got stuck? Why an old NIFI=C2=A0 cluster = conf works better than a new NIFI? > Hope you can help me with this.=C2=A0 > Thanks > =C2=A0 -- This message was sent by Atlassian Jira (v8.3.4#803005)