From dev-return-20993-archive-asf-public=cust-asf.ponee.io@nifi.apache.org Mon May 4 16:46:05 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4BB06180608 for ; Mon, 4 May 2020 18:46:05 +0200 (CEST) Received: (qmail 71612 invoked by uid 500); 4 May 2020 16:46:04 -0000 Mailing-List: contact dev-help@nifi.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@nifi.apache.org Delivered-To: mailing list dev@nifi.apache.org Delivered-To: moderator for dev@nifi.apache.org Received: (qmail 62109 invoked by uid 99); 4 May 2020 16:40:56 -0000 Subject: Re: CSVRecordSetWriter Flow File Content is Empty From: james medel X-Mailer: LuaSocket 3.0-rc1 MIME-Version: 1.0 To: References: x-ponymail-agent: PonyMail Composer/0.2 Content-Type: text/plain; charset=utf-8 Message-ID: Date: Mon, 04 May 2020 16:40:54 -0000 x-ponymail-sender: f842a7b12d5f015c02da8aa863b23750432596a0 In-Reply-To: Hi Mark, Thank you for your response. Yes it turned out the issue was related to my schema didn't match the record. So, I changed the CSVRecordSetWriter to "Inherit Record Schema" and that worked fine. I was able to get the expected results in the output flow file. On 2020/04/29 16:44:28, Mark Payne wrote: > James, > > It’s hard to say without having your code, but generally if you’re seeing the output not containing any information, it means that the schema doesn’t match the Record given to it. The best approach is probably to just set a breakpoint in the CSVRecordSetWriter so that you can see the incoming Record object and see which fields it’s attempting to pull from the Record to understand why it’s not writing anything out. > > Thanks > -Mark > > > > On Apr 29, 2020, at 12:29 PM, james medel wrote: > > > > Hi I have been working on a custom processor "ExecuteMojoScoringRecord" that integrates H2O MOJO Scoring Pipeline with NiFi to do batch scoring and real time scoring on test data to get predicted label(s). In an earlier test of the processor, I used a hydraulic system sensor data set and only predicted 1 label, which was cooling efficiency. The flow file outputted by the "ExecuteMojoScoringRecord" had content of 1 predicted label field name and field value. > > > > However, later when I updated the Hydraulic data set by adding a cooling condition label as the last column, now the processor's MOJO was predicting cooling condition, which returns 3 predicted labels ""cool_cond_y_3, cool_cond_y_20, cool_cond_y_100" and their associated prediction values. However, the flow files coming out of this processor have empty content. The prediction values are missing. Right before the writer.write(scoredFirstRecord), I added logs into the code to see if the key value pairs of the scoredFirstRecord contained the field names and field values. The data showed up in the logs. Yet, it does not show up the flow file content. The changes I made to the output Avro schema were adding 3 field names and their field data types, which are double cause the prediction data type is Float64. Why do you think the flow file content is empty? > > > >