nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otto Fowler <>
Subject Re: Grok Reader, Extract and the default patterns
Date Wed, 09 May 2018 01:37:41 GMT
I would add a test for putting in the defaults as a file when they are
added by the processor as well.


I could expose a new property to turn it on.  But really, by rule of least
surprise, I would expect the core groks to always
exist and to be able to overwrite if necessary.  We do that in metron.

There is something that feels wrong about possible duplication between the
reader serialization service nar and standard processors (
GrokExpressionValidator for example )
but that is another kettle of fish.

On May 8, 2018 at 21:33:59, Otto Fowler ( wrote:

I think so, the way that java-grok works ( refactored for 0.1.9 ) is that
you register
sets of patterns with the compiler, and the compiler produces the Grok

When registering things with the same name, the last one in wins, so I
think if an existing
user had put a pattern file in the configuration with patterns named the
same as the
defaults, then they would just replace them in the map.

On May 8, 2018 at 21:24:35, Joe Witt ( wrote:

You're probably not missing anything. ExtractGrok came before the
record reader/writer enlightenment phase.

If you think ExtractGrok is useful and want to improve it as you
suggest I think you're good to go provided you do so in a way that
doesn't break existing flows (change their behavior unless they opt in
so to speak). Is that feasible for your idea?


On Tue, May 8, 2018 at 9:11 PM, Otto Fowler <> wrote:
> I’m working on upgrading java-grok to the new 0.1.9 release. While going
> through the GrokReader the the ExtractGrok components I noticed that they
> differ in a very important way grok wise.
> The reader loads the default patterns ( which are a copy of the ubiquitous
> default patterns in java-grok itself. The older ( I assume ) ExtractGrok
> processor does not.
> I’m wondering if there is a reason for this going back to the creation of
> the processor. It seems to me it would be better for the ExtractGrok
> processor and the GrokReader to work similarly with regards
> to the default patterns. At the moment, there is only one pattern file
> allowed to be specified with ExtractGrok ( I think there is PR for
> pattern files ). Besides consistency between the two components,
> it seems like it would be better for users if they didn’t have to waste or
> merge pattern files to get the common patterns.
> I apologize in advance if I am missing something here.
> Would anyone object to setting the default patterns as we do in the
> GrokReader?
> ottO

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message