nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From james medel <jamesmede...@gmail.com>
Subject Re: CSVRecordSetWriter Flow File Content is Empty
Date Mon, 04 May 2020 16:40:54 GMT
Hi Mark,

Thank you for your response. Yes it turned out the issue was related to my schema didn't match
the record. So, I changed the CSVRecordSetWriter to "Inherit Record Schema" and that worked
fine. I was able to get the expected results in the output flow file.

On 2020/04/29 16:44:28, Mark Payne <markap14@hotmail.com> wrote: 
> James,
> 
> It’s hard to say without having your code, but generally if you’re seeing the output
not containing any information, it means that the schema doesn’t match the Record given
to it. The best approach is probably to just set a breakpoint in the CSVRecordSetWriter so
that you can see the incoming Record object and see which fields it’s attempting to pull
from the Record to understand why it’s not writing anything out.
> 
> Thanks
> -Mark
> 
> 
> > On Apr 29, 2020, at 12:29 PM, james medel <jamesmedel94@gmail.com> wrote:
> > 
> > Hi I have been working on a custom processor "ExecuteMojoScoringRecord" that integrates
H2O MOJO Scoring Pipeline with NiFi to do batch scoring and real time scoring on test data
to get predicted label(s). In an earlier test of the processor, I used a hydraulic system
sensor data set and only predicted 1 label, which was cooling efficiency. The flow file outputted
by the "ExecuteMojoScoringRecord" had content of 1 predicted label field name and field value.
> > 
> > However, later when I updated the Hydraulic data set by adding a cooling condition
label as the last column, now the processor's MOJO was predicting cooling condition, which
returns 3 predicted labels ""cool_cond_y_3, cool_cond_y_20, cool_cond_y_100" and their associated
prediction values. However,  the flow files coming out of this processor have empty content.
The prediction values are missing. Right before the writer.write(scoredFirstRecord), I added
logs into the code to see if the key value pairs of the scoredFirstRecord contained the field
names and field values. The data showed up in the logs. Yet, it does not show up the flow
file content. The changes I made to the output Avro schema were adding 3 field names and their
field data types, which are double cause the prediction data type is Float64. Why do you think
the flow file content is empty?
> > 
> 
> 

Mime
View raw message