From issues-return-92637-archive-asf-public=cust-asf.ponee.io@nifi.apache.org Fri Feb 21 19:36:02 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 99638180657 for ; Fri, 21 Feb 2020 20:36:02 +0100 (CET) Received: (qmail 34427 invoked by uid 500); 21 Feb 2020 19:36:02 -0000 Mailing-List: contact issues-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@nifi.apache.org Delivered-To: mailing list issues@nifi.apache.org Received: (qmail 34413 invoked by uid 99); 21 Feb 2020 19:36:02 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 21 Feb 2020 19:36:02 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 20E8BE2F0E for ; Fri, 21 Feb 2020 19:36:01 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 394F3780254 for ; Fri, 21 Feb 2020 19:36:00 +0000 (UTC) Date: Fri, 21 Feb 2020 19:36:00 +0000 (UTC) From: "Davy Alvarenga Machado (Jira)" To: issues@nifi.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (NIFI-4239) Implement a CaptureChangePostgreSQL processor MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/NIFI-4239?page=3Dcom.atlassian.= jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D17042= 130#comment-17042130 ]=20 Davy Alvarenga Machado commented on NIFI-4239: ---------------------------------------------- Hi [~shoor]! We already did the pull request on Apache NiFi repository: [https://github.com/apache/nifi/pull/4065] =C2=A0 > Implement a CaptureChangePostgreSQL processor > --------------------------------------------- > > Key: NIFI-4239 > URL: https://issues.apache.org/jira/browse/NIFI-4239 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions > Reporter: Gerdan Santos > Assignee: Gerdan Santos > Priority: Major > Attachments: 2020-01-11_23-37-172.png, 2020-01-11_23-37-58.png > > Original Estimate: 2,688h > Time Spent: 1h 10m > Remaining Estimate: 2,686h 50m > > Inspired on CaptureChangeMySQL, this processor can use one Streaming Repl= ication PostgreSQL Connection to allow access to their transactional logs a= nd such, in order for external clients to have a "change data capture" (CDC= ) capability. > The processor would include properties needed for PostgreSQL connectivity= PostgreSQL Streaming Replication. It would also need to keep a "sequence I= D" such that an EnforceOrder processor (NIFI-3414) for example could guaran= tee the order of CDC events for use cases such as replication.=20 > It will likely need State Management for that, and may need other facilit= ies such as a DistributedMapCache in order to keep information (column name= s and types, e.g.) that enrich the raw CDC events. > The processor would accept no incoming connections (it is a "get" or sour= ce processor), would be intended to run on the primary node only as a singl= e threaded processor, and would generate a flow file for each operation (IN= SERT, UPDATE, DELETE, e.g.) in one or some number of formats (JSON, e.g.). -- This message was sent by Atlassian Jira (v8.3.4#803005)