From user-return-25170-archive-asf-public=cust-asf.ponee.io@flink.apache.org Mon Jan 7 05:06:15 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 2B0CE180636 for ; Mon, 7 Jan 2019 05:06:15 +0100 (CET) Received: (qmail 6684 invoked by uid 500); 7 Jan 2019 04:06:12 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 6672 invoked by uid 99); 7 Jan 2019 04:06:12 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Jan 2019 04:06:12 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 3E85D180EE0 for ; Mon, 7 Jan 2019 04:06:12 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.654 X-Spam-Level: * X-Spam-Status: No, score=1.654 tagged_above=-999 required=6.31 tests=[DKIMWL_WL_MED=-0.145, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id rRcS2KUFF8Q8 for ; Mon, 7 Jan 2019 04:06:11 +0000 (UTC) Received: from mail-ed1-f41.google.com (mail-ed1-f41.google.com [209.85.208.41]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id D552960E10 for ; Mon, 7 Jan 2019 04:06:10 +0000 (UTC) Received: by mail-ed1-f41.google.com with SMTP id g22so36610332edr.7 for ; Sun, 06 Jan 2019 20:06:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=1qcIewrxZd9BfidUWz2oeyjEykZ/O46h2CY1p9t0tpY=; b=V10ADH1FhP6/f7ZXMZ3IO7wNtLv1XrXe52uYCeeUmhDw59+0hmOqjfrXzz9GGCPBQz z6SuSvRJM4w1cAh+hjKnhFak7NvWHlkXIR16Et+0Gtzwv3W4iIv6/zCSZp5lJKfQVWZ8 PVtJDdnSVywX0l2F+DhsCj1Q28O93/cHvfTKfYLbLfRDGSziZ845IsPvHmZVtior1/9P Sk8vR7a+zAiCbpFG9S6OmFWTWpnO7OXpzF4WSts7/i74hx+Rzgo3osJ2Gk7kn0qSOoMN mVPQdr0LKqjLrg3RWiP+FoD3QJqotDq4YmyL+FIImZ33IENBuAODWDVbyXsdS7Vit+d9 XkhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=1qcIewrxZd9BfidUWz2oeyjEykZ/O46h2CY1p9t0tpY=; b=lCVVw8snJoLYKhfUTxEvyrxx0tBo1eKA0Gr37UlH2DI5F3xl5q/dcY0il7J+ko1j2O oUgTRBj2nPYllWge7O6toazYy1ukKf9+0w66fmBT2jRlneaKhm03gzZCLQi/Bw5WwT73 W3dWb+CxR0JuPBcRAcTGpDM/mZUWHmGRE1goN7ZcR3GYAa7IxuibeihXJjxQYJaxlhuV 4sGlpjmSIKNsoQtVTAz9HI+TphDIT1CFox9F/Fa2HWgDdQsiQAWjCPerCxxJrg/iqhhf MFJRYDCCL+/9ooZJ7j0dKPzjul+qdg2mKi+0bR2W7FxwwgbZclWXSiqt9ADLQPN3/g8u BoLQ== X-Gm-Message-State: AJcUukf9p3wNebdk/CnbHmLScsPF4K9xLwOzlXDQ6sMbsL+ilWoVX3Qf Vvm7YudHUnVCskgXq5GmzL3lZB4W9Jv9Lvyrq6O59A== X-Google-Smtp-Source: ALg8bN6u+aNOGqLBjhygk09GXz0pm+EAZ+eAk80nFgRxKoI/E/2IkxnasoPF84HZXONC6sjfyh/6s59OuRfuul5xZQw= X-Received: by 2002:a17:906:a384:: with SMTP id k4-v6mr9685112ejz.243.1546833963827; Sun, 06 Jan 2019 20:06:03 -0800 (PST) MIME-Version: 1.0 From: Gagan Agrawal Date: Mon, 7 Jan 2019 09:35:50 +0530 Message-ID: Subject: Buffer stats when Back Pressure is high To: user Content-Type: multipart/alternative; boundary="000000000000f0c5a6057ed65816" --000000000000f0c5a6057ed65816 Content-Type: text/plain; charset="UTF-8" Hi, I want to understand does any of buffer stats help in debugging / validating that downstream operator is performing slow when Back Pressure is high? Say I have A -> B operators and A shows High Back Pressure which indicates something wrong or not performing well on B side which is slowing down operator A. However when I look at buffers.inputQueueLength for operator B, it's 0. My understanding is that when B is processing slow, it's input buffer will be full of incoming messages which ultimately blocks/slows down upstream operator A. However it doesn't seem to be happening in my case. Can someone throw some light on how should different stats around buffers (e.g buffers.inPoolUsage, buffers.inputQueueLength, numBuffersInLocalPerSecond, numBuffersInRemotePerSecond) look like when downstream operator is performing slow? Gagan --000000000000f0c5a6057ed65816 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,
I want to understand does any of buffer stats help= in debugging / validating that downstream operator is performing slow when= Back Pressure is high? Say I have A -> B operators and A shows High Bac= k Pressure which indicates something wrong or not performing well on B side= which is slowing down operator A. However when I look at buffers.inputQueu= eLength for operator B, it's 0. My understanding is that when B is proc= essing slow, it's input buffer will be full of incoming messages which = ultimately blocks/slows down upstream operator A. However it doesn't se= em to be happening in my case. Can someone throw some light on how should d= ifferent stats around buffers (e.g buffers.inPoolUsage, buffers.inputQueueL= ength, numBuffersInLocalPerSecond, numBuffersInRemotePerSecond) look like w= hen downstream operator is performing slow?

Gagan<= /div>
--000000000000f0c5a6057ed65816--