nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy LoPresto <alopre...@apache.org>
Subject Re: CSVRecordSetWriter Flow File Content is Empty
Date Wed, 29 Apr 2020 19:36:32 GMT
Hi James,

Mark’s approach is right. Again, without seeing your code, I would throw out that a common
issue in custom processor development is performing the session.write(), session.putAttribute(),
etc. operations but failing to _assign the result back to the flowfile variable_. The variable
you pass to the output relationships has to have the “most current” version of the flowfile
to result in the modified content being successfully transferred. If you’re able to share
the processor code, we can probably help diagnose further. 


Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
He/Him
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Apr 29, 2020, at 9:44 AM, Mark Payne <markap14@hotmail.com> wrote:
> 
> James,
> 
> It’s hard to say without having your code, but generally if you’re seeing the output
not containing any information, it means that the schema doesn’t match the Record given
to it. The best approach is probably to just set a breakpoint in the CSVRecordSetWriter so
that you can see the incoming Record object and see which fields it’s attempting to pull
from the Record to understand why it’s not writing anything out.
> 
> Thanks
> -Mark
> 
> 
>> On Apr 29, 2020, at 12:29 PM, james medel <jamesmedel94@gmail.com> wrote:
>> 
>> Hi I have been working on a custom processor "ExecuteMojoScoringRecord" that integrates
H2O MOJO Scoring Pipeline with NiFi to do batch scoring and real time scoring on test data
to get predicted label(s). In an earlier test of the processor, I used a hydraulic system
sensor data set and only predicted 1 label, which was cooling efficiency. The flow file outputted
by the "ExecuteMojoScoringRecord" had content of 1 predicted label field name and field value.
>> 
>> However, later when I updated the Hydraulic data set by adding a cooling condition
label as the last column, now the processor's MOJO was predicting cooling condition, which
returns 3 predicted labels ""cool_cond_y_3, cool_cond_y_20, cool_cond_y_100" and their associated
prediction values. However,  the flow files coming out of this processor have empty content.
The prediction values are missing. Right before the writer.write(scoredFirstRecord), I added
logs into the code to see if the key value pairs of the scoredFirstRecord contained the field
names and field values. The data showed up in the logs. Yet, it does not show up the flow
file content. The changes I made to the output Avro schema were adding 3 field names and their
field data types, which are double cause the prediction data type is Float64. Why do you think
the flow file content is empty?
>> 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message