From dev-return-18435-archive-asf-public=cust-asf.ponee.io@nifi.apache.org Fri Nov 2 03:38:26 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 45E0F180652 for ; Fri, 2 Nov 2018 03:38:26 +0100 (CET) Received: (qmail 89274 invoked by uid 500); 2 Nov 2018 02:38:25 -0000 Mailing-List: contact dev-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@nifi.apache.org Delivered-To: mailing list dev@nifi.apache.org Received: (qmail 89245 invoked by uid 99); 2 Nov 2018 02:38:24 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Nov 2018 02:38:24 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id D48DF182D7B for ; Fri, 2 Nov 2018 02:38:23 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.888 X-Spam-Level: X-Spam-Status: No, score=0.888 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_REPLY=1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id vn-YLH6uWKSa for ; Fri, 2 Nov 2018 02:38:22 +0000 (UTC) Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 066725F530 for ; Fri, 2 Nov 2018 02:38:22 +0000 (UTC) Received: by mail-qk1-f172.google.com with SMTP id f18-v6so877426qkm.7 for ; Thu, 01 Nov 2018 19:38:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=bgO2puytD5fcrgEeE9gMAifsxDw6YtoHcglKlO1trJA=; b=klYWK/1QVkh/4WASem6fb2KWljpzdM06e1wZMYhO88tH3WfnpyTBHBQrWlTzn8E7Jj MXEM3ickFFUrA6kOKadlpP8Wo2UnF+O/E0VtMuFgRu65pOwpeth5HXYZ7EEi8f37Qzl2 Qj9eNHr1f9FXRSxS1QRvqMUY7CJNOzJvlqJhBXrgHo0g2kRVBKGHw/ga5i1pYmScpTg2 bLlKFj0uGHslbKx7m78ViNMS4a8a7rt4yFjcnACQFpXDZTwuWewznFa55kxSzY9NR1qk kOTsu3wB9duQ5/uz5EpLDw50YFUghyK6ME0jfQ5s6Hawhf1G3m4njIsjRsLGKnuqh87W Z/bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=bgO2puytD5fcrgEeE9gMAifsxDw6YtoHcglKlO1trJA=; b=pHTuQ+uWaaZ9OB5+prKY4XV7P/XwddaTxlTMLB7d51+dmGdvIzsb8bxzAXjfBjI7aV j9E8elLrcbJLhPDEQNwKB05ShFDCS+ghqzmkBfdWOZCnf8znOBFItWRYaTSD1wd7iGc5 Et8K3TRZZopg+SsjUH/bMVjaMqeVYPL54+n3AGfm0oKOC4bdUDF0hxvm+6Xo+/2Vn2pF 29dD4rxmRC+ASh9P7RebvIFCt/I7y0SqCd5BdiM3EFXDjm4TinjznXWRgqqHUBaLDGVB 2rihVJj5ltBL3NnGvxX/7YAKW57CzqLKSiVTizaVmTgtwOlrGF/IzrrK8f2/vGErOmYq 2jzQ== X-Gm-Message-State: AGRZ1gL2yZMDKMYTSAsx2NSDpliSAQubEp7gS6/kH4029sAWD7OBLhsz PnFDyuPJbvxWkVv9VVsY0haSVcJyL1oeRzRE6acWmFkf X-Google-Smtp-Source: AJdET5evT8U76zbUGhcV+/rJF2c1pZInvqCTH6ntJeb2XpPjMwnmMM0wslS43HX/g2j8OEcKo3VDpTT0b41frt/GUTY= X-Received: by 2002:a37:dd43:: with SMTP id n64mr9297755qki.7.1541126300666; Thu, 01 Nov 2018 19:38:20 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Koji Kawamura Date: Fri, 2 Nov 2018 11:38:09 +0900 Message-ID: Subject: Re: Load Balancing To: dev Content-Type: text/plain; charset="UTF-8" Hi Mark, > In this scenario, should the nifi.cluster.load.balance.comms.timeout have caused the balancing operation to terminate (unsuccessful)? I agree with that. Wasn't there any WARN log messages written? Currently NiFi UI doesn't have capability to show load-balancing related error on the canvas, other than currently active or not. > Another question: the usage of nifi.cluster.load.balance.host (and .port) values is not clear to me. If Node A set's this value for Node B's FQDN, would this allow Node A to "spoof" Node B and accept load balanced items intended for Node B? The two properties are used how a NiFi node opens its socket to receive load-balanced data from other node. It can be useful when a node have different NICs and you want to use specific one for load-balancing. If you specify a hostname or ip-address that is not for the node, then you'll get an exception when the node tries opening a socket since it can't bind to the specified address. Thanks, Koji On Thu, Nov 1, 2018 at 3:04 AM mark.o.bean@gmail.com wrote: > > I found the problem: iptables was blocking the load balancing port. Once the port was opened, the balance completed and all files were visible via List queue. > > In this scenario, should the nifi.cluster.load.balance.comms.timeout have caused the balancing operation to terminate (unsuccessful)? > > Another question: the usage of nifi.cluster.load.balance.host (and .port) values is not clear to me. If Node A set's this value for Node B's FQDN, would this allow Node A to "spoof" Node B and accept load balanced items intended for Node B? > > > On 2018/10/31 17:45:29, Mark Bean wrote: > > I am trying to understand how the load balancing works in NiFi 1.8.0. > > > > I have a 2-node Cluster. I set an UpdateAttribute to set the value of a > > property, "balancer", to either 0 or 1. I am using stateful EL for this: > > ${getStateValue('balancer'):plus(1):mod(2)}. > > > > The connection for the output of the UpdateAttribute processor is load > > balanced. > > Load Balance Strategy: Partition by attribute > > Attribute name: balancer > > > > The queue in the connection contains 5 objects, but when I perform a "List > > queue", I only see 3 flowfiles. All of the flowfiles are on the same Node, > > and as expected have the same "balancer" attribute value. > > > > Presumably, the other 2 flowfiles were load-balanced to the other Node. > > However, they should still be visible in List queue, correct? > > > > Perhaps related, the load balance icon on the connection indicates > > "Actively balancing...". There are only two 10 byte files, but the > > balancing never seems to complete. > >