nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ed J <e...@protonmail.com.INVALID>
Subject Re: POJO to Record
Date Tue, 02 Oct 2018 01:12:37 GMT
Hi Matt,
Thanks for the tips.  Will look into those options this week.

Many of the fields are dynamic and our XML schema doesn't really qualify for XMLReader based
on what I can glean from the docs, for example: "Records are expected in the second level
of XML data, embedded in an enclosing root tag."

The other rationale for building an external custom parser to transform the XML is that it
provides the flexibility to parse from other tools/apps in addition to NiFi.

-Ed

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Friday, September 28, 2018 11:49 AM, Matt Burgess <mattyb149@apache.org> wrote:

> Moving users list to BCC and adding devs list, as this question is
> better suited for the devs.
>
> Ed,
>
> Does XMLReader not parse your XML? I guess you'd need to know the
> schema so if you don't (especially if it is dynamic) then you might be
> faced with a custom processor as you described.
>
> Take a look at DataTypeUtils, it has methods toRecord() and
> convertRecordFieldtoObject() which helps you convert from POJO
> <->Record. The JoltTransformRecord processor uses these, so you can
> have a look at that code for an example.
>
> Regards,
> Matt
>
> On Fri, Sep 28, 2018 at 11:28 AM Ed J edja@protonmail.com wrote:
>
> > Hi,
> > I'm ingesting some XML with an overly complicated structure and I've build a custom
java DOM parser to wrangle it into a saner form. (Is that frowned upon - should I just use
the built-in processors to wrangle it?)
> > So my question is, I've parsed the XML into a simple POJO, how do I get that pojo
into the next processor as a 'Record'. My custom nifi processor's onTrigger looks like this:
> > MyParser parser = new MyParser();
> > MyPojo pojo = parser.parse(flowFileContents);
> > // TODO: convert pojo to record
> > Record myrecord = pojo.toRecord(); // how to do this?
> > session.transfer(myrecord, REL_SUCCESS)
> > I'm guessing I could convert the pojo to JSON, write that out to the next stage,
and then use a JsonRecordReader to convert the JSON into a record. However, if I can go straight
from pojo to record, that seems more efficient - just not sure how to go about it. Thanks.



Mime
View raw message