From commits-return-19846-apmail-nifi-commits-archive=nifi.apache.org@nifi.apache.org Mon Feb 1 16:56:50 2016 Return-Path: X-Original-To: apmail-nifi-commits-archive@minotaur.apache.org Delivered-To: apmail-nifi-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id F3D9C18ACE for ; Mon, 1 Feb 2016 16:56:49 +0000 (UTC) Received: (qmail 25973 invoked by uid 500); 1 Feb 2016 16:56:40 -0000 Delivered-To: apmail-nifi-commits-archive@nifi.apache.org Received: (qmail 25930 invoked by uid 500); 1 Feb 2016 16:56:40 -0000 Mailing-List: contact commits-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@nifi.apache.org Delivered-To: mailing list commits@nifi.apache.org Received: (qmail 25900 invoked by uid 99); 1 Feb 2016 16:56:40 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Feb 2016 16:56:40 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 29B252C1F6C for ; Mon, 1 Feb 2016 16:56:40 +0000 (UTC) Date: Mon, 1 Feb 2016 16:56:40 +0000 (UTC) From: "ASF subversion and git services (JIRA)" To: commits@nifi.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (NIFI-1093) ConvertJSONToSQL incorrectly detects required columns MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/NIFI-1093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15126557#comment-15126557 ] ASF subversion and git services commented on NIFI-1093: ------------------------------------------------------- Commit 171b9c4e94113dc6e3c42154eb500e046620249e in nifi's branch refs/heads/master from [~Daniel Cave] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=171b9c4 ] NIFI-1093 - Added settings for processing JSON documents when validating against database columns to fail, log a warning, or ignore unmatched columns. Added tests for new settings. Signed-off-by: Mark Payne > ConvertJSONToSQL incorrectly detects required columns > ----------------------------------------------------- > > Key: NIFI-1093 > URL: https://issues.apache.org/jira/browse/NIFI-1093 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions > Reporter: Randy Gelhausen > Assignee: Oleg Zhurakousky > Fix For: 0.5.0 > > Attachments: 0001-NIFI-1093-Added-settings-for-processing-JSON-documen.patch > > > {code} > create table device_pings(id varchar not null, ts varchar not null, bssid varchar, ssid varchar, noise integer, signal integer, constraint pk primary key (id, ts)) > {code} > With the example DDL above, neither SSID nor BSSID are required fields, yet ConvertJSONToSQL throws an exception if the input JSON lacks those fields: > {code} > 2015-11-01 17:47:10,373 ERROR [Timer-Driven Process Thread-6] o.a.n.p.standard.ConvertJSONToSQL ConvertJSONToSQL[id=a336eb2b-fc63-4118-b098-c0ded1dd5520] Failed to convert StandardFlowFileRecord[uuid=5d2c05f0-982e-4feb-94b1-d9946be730d4,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1446416358991-2, container=default, section=2], offset=203306, length=614],offset=0,name=1446418019645796000,size=614] to a SQL INSERT statement due to org.apache.nifi.processor.exception.ProcessException: JSON does not have a value for the Required column 'BSSID'; routing to failure: org.apache.nifi.processor.exception.ProcessException: JSON does not have a value for the Required column 'BSSID' > 2015-11-01 17:47:10,381 ERROR [Timer-Driven Process Thread-6] o.a.n.p.standard.ConvertJSONToSQL ConvertJSONToSQL[id=a336eb2b-fc63-4118-b098-c0ded1dd5520] Failed to convert StandardFlowFileRecord[uuid=727b8c3c-66c5-4d6a-8cdc-602da8b80132,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1446416358991-2, container=default, section=2], offset=203920, length=674],offset=0,name=1446418019645796000,size=674] to a SQL INSERT statement due to org.apache.nifi.processor.exception.ProcessException: JSON does not have a value for the Required column 'SSID'; routing to failure: org.apache.nifi.processor.exception.ProcessException: JSON does not have a value for the Required column 'SSID' > {code} > The processor has "Unmatched Field Behavior" property. Should there be an additional property "Unmatched Column Behavior" property that allows SQL statements to be generated from the set of columns actually available in the JSON? -- This message was sent by Atlassian JIRA (v6.3.4#6332)