From dev-return-111792-archive-asf-public=cust-asf.ponee.io@cloudstack.apache.org Tue Aug 14 07:50:58 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id B9CCD180654 for ; Tue, 14 Aug 2018 07:50:57 +0200 (CEST) Received: (qmail 29283 invoked by uid 500); 14 Aug 2018 05:50:56 -0000 Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cloudstack.apache.org Delivered-To: mailing list dev@cloudstack.apache.org Received: (qmail 29270 invoked by uid 99); 14 Aug 2018 05:50:55 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Aug 2018 05:50:55 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 359191A28A5 for ; Tue, 14 Aug 2018 05:50:55 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.989 X-Spam-Level: * X-Spam-Status: No, score=1.989 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=bw-sw-com.20150623.gappssmtp.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id VLVvoFTDwQtB for ; Tue, 14 Aug 2018 05:50:51 +0000 (UTC) Received: from mail-it0-f52.google.com (mail-it0-f52.google.com [209.85.214.52]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id E616B5F3EF for ; Tue, 14 Aug 2018 05:50:50 +0000 (UTC) Received: by mail-it0-f52.google.com with SMTP id h23-v6so17942386ita.5 for ; Mon, 13 Aug 2018 22:50:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bw-sw-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=XwXgq9Gbx09SB2MkCN7xx0bnsVojG040r2iwPq5Ip20=; b=V9nTV7OgkJDTYAysceD7zKxeAAkbOv3mwOiMQHll9CuB4+BJs3F113IKwWGCWywi2V 8FnPdooD9fLItauAVWFmqawo4YwYtA7Ji+JWrh89RT0T6PprZ+icceYPROJJ2iqAZH5t PwspEjNj8SIGB8M0PbdXHQk/l2sl+4IrfYxLeHxaDE8Ry7dA74jD8MWp5IeAxUL/o0rO q5g6MJP7/Dkqqrs63e0A6njMWcCJVeCgvSQWFllKpcJo5mi4yClJHmKhhmRGjcRgV1Gy 9xs4jYSsSN4fA1+mBKlqpixMM041Z8Bt8Y94gpikL4PIPS7t/o0UQrLk6ukQfnPfwRVr LvkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=XwXgq9Gbx09SB2MkCN7xx0bnsVojG040r2iwPq5Ip20=; b=fcP903KtPk4CuyUJ3CPNhSDOP290Jrf3Ew/FTOLFpF9P0vffn6oWLFhy32jbfwtwmQ 4v6kx3J1zaxRkhhCmB1t5UwKORmANR17O11nkmbQZ2F0jvikid3b8lFY2zmfSfcwW8YG VhsMG6twyMSuvDEAZuockmFp9HV8EeYNeOJUTAt7Sx37J69mCeuPqPt8fyYU0eKBv0yu uNsUOEYLeF/ssgQXJM7x6C2Znn5z7DK9KNzzJ1MHQhXxNtiFJhaZoMLg/jd21HSn2Oz/ asjsf4kV7cSe8WycAY9ZPmKmVLM9bKuVo9R+xS/Ca7uNJx468nn4U+TI5IZ/ZfpOj5Q3 S51Q== X-Gm-Message-State: AOUpUlFOsG4ldl8YFRPHpQMEe/BRBw7D9Dol8XDnEuqxoEvN0OBiseAv 5ttXyt+o8OOXb86oQhN0Aozb3zgRKWH/5t1sfBduJA== X-Google-Smtp-Source: AA+uWPzpFOXX91onVMbfQTXePC7iLa1IW4I1IAMVO+K+6lS6E47vw7YiB3YqdFVGPfcrOOzv0ATaHqHdCzucPgvRzwI= X-Received: by 2002:a24:6a8a:: with SMTP id l132-v6mr13882757itc.50.1534225843809; Mon, 13 Aug 2018 22:50:43 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a6b:fe12:0:0:0:0:0 with HTTP; Mon, 13 Aug 2018 22:50:43 -0700 (PDT) From: Ivan Kudryavtsev Date: Tue, 14 Aug 2018 12:50:43 +0700 Message-ID: Subject: Hybrid Event Bus for CloudStack (for CloudStack users who use Kafka or RMQ) To: users , dev Content-Type: multipart/alternative; boundary="0000000000006d065e05735eca07" --0000000000006d065e05735eca07 Content-Type: text/plain; charset="UTF-8" Hello, users, devs. Today we would like to share new event bus plug-in (hybrid) which combines inMemory (default) and (Kafka or RabbitMQ) event buses, so inMemory is used for intra-CloudStack purposes and RMQ or Kafka are used for external. We use Kafka Event Bus in our deployment and after the development of a new plug-in (which we publish soon), we discovered that this bus has lack of functionality. Internal event receivers are not able to subscribe to events when that bus is used. Actually, it's quite a nasty thing because of cutting certain CloudStack functionality and It's really impossible who would like to subscribe to events even inside the CloudStack core, e.g. to implement delayed processing. So, using standard Kafka Event Bus these functions stop working and the regression discovery will not be handy. I know that RabbitMQ implementation also has certain subscription problems. To overcome it, we implemented a "hybrid" even bus, which uses inMemory for internals and any other for external communications. You can find that bus on GitHub: https://github.com/bwsw/cloud-plugin-event-bus-hybrid License: Apache 2 If the developers and community wish to merge it into the main tree, please feel free, we grant it. We also can merge ourselves if project leaders find the question worth it. -- With best regards, Ivan Kudryavtsev Bitworks LLC Cell: +7-923-414-1515 WWW: http://bitworks.software/ --0000000000006d065e05735eca07--