nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-360) Create Processors to work against JSON data
Date Sun, 01 Mar 2015 01:35:05 GMT

    [ https://issues.apache.org/jira/browse/NIFI-360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14341879#comment-14341879
] 

ASF GitHub Bot commented on NIFI-360:
-------------------------------------

Github user apiri commented on a diff in the pull request:

    https://github.com/apache/incubator-nifi/pull/28#discussion_r25563122
  
    --- Diff: nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/JsonUtils.java
---
    @@ -0,0 +1,127 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *     http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +package org.apache.nifi.processors.standard.util;
    +
    +import com.jayway.jsonpath.Configuration;
    +import com.jayway.jsonpath.DocumentContext;
    +import com.jayway.jsonpath.InvalidPathException;
    +import com.jayway.jsonpath.JsonPath;
    +import com.jayway.jsonpath.spi.json.JsonProvider;
    +import net.minidev.json.JSONValue;
    +import org.apache.nifi.components.ValidationContext;
    +import org.apache.nifi.components.ValidationResult;
    +import org.apache.nifi.components.Validator;
    +import org.apache.nifi.flowfile.FlowFile;
    +import org.apache.nifi.processor.ProcessSession;
    +import org.apache.nifi.processor.io.InputStreamCallback;
    +import org.apache.nifi.stream.io.BufferedInputStream;
    +import org.apache.nifi.util.BooleanHolder;
    +import org.apache.nifi.util.ObjectHolder;
    +
    +import java.io.IOException;
    +import java.io.InputStream;
    +import java.io.InputStreamReader;
    +import java.util.List;
    +import java.util.Map;
    +
    +/**
    + * Provides utilities for interacting with JSON elements and JsonPath expressions and
results
    + *
    + * @see <a href="http://json.org">http://json.org</a>
    + * @see <a href="https://github.com/jayway/JsonPath">https://github.com/jayway/JsonPath</a>
    + */
    +public class JsonUtils {
    +
    +    static final JsonProvider JSON_PROVIDER = Configuration.defaultConfiguration().jsonProvider();
    +
    +    public static final Validator JSON_PATH_VALIDATOR = new Validator() {
    +        @Override
    +        public ValidationResult validate(final String subject, final String input, final
ValidationContext context) {
    +            String error = null;
    +            try {
    +                JsonPath compile = JsonPath.compile(input);
    +            } catch (InvalidPathException ipe) {
    +                error = ipe.toString();
    +            }
    +            return new ValidationResult.Builder().valid(error == null).explanation(error).build();
    +        }
    +    };
    +
    +    public static DocumentContext validateAndEstablishJsonContext(ProcessSession processSession,
FlowFile flowFile) {
    +
    +        final BooleanHolder validJsonHolder = new BooleanHolder(false);
    +        processSession.read(flowFile, new InputStreamCallback() {
    +            @Override
    +            public void process(InputStream in) throws IOException {
    +                validJsonHolder.set(JsonUtils.isValidJson(in));
    +            }
    +        });
    +
    +        // Parse the document once into an associated context to support multiple path
evaluations if specified
    +        final ObjectHolder<DocumentContext> contextHolder = new ObjectHolder<>(null);
    +
    +        if (validJsonHolder.get()) {
    +            processSession.read(flowFile, new InputStreamCallback() {
    --- End diff --
    
    Yeah, looking at what that method is actually doing is just catching an exception when
parsing it instead of just inspecting.  Good catch.  Have to see what the API provides, but
seems like it is not buying much.


> Create Processors to work against JSON data
> -------------------------------------------
>
>                 Key: NIFI-360
>                 URL: https://issues.apache.org/jira/browse/NIFI-360
>             Project: Apache NiFi
>          Issue Type: New Feature
>          Components: Extensions
>            Reporter: Aldrin Piri
>            Assignee: Aldrin Piri
>            Priority: Minor
>              Labels: processor
>
> I have created two Processors, EvaluateJsonPath and SplitJson which are analogs of the
functionality provided through EvaluateXPath and SpiltXML.
> Both are powered primarily around the usage of [JsonPath by Jayway|https://github.com/jayway/JsonPath].
> Their capability descriptions are provided below:
> {panel:title= EvaluateJsonPath}
> Evaluates one or more JsonPath expressions against the content of a FlowFile.  The results
of those expressions are assigned to FlowFile Attributes or are written to the content of
the FlowFile itself, depending on configuration of the Processor. JsonPaths are entered by
adding user-defined properties; the name of the property maps to the Attribute Name into which
the result will be placed (if the Destination is flowfile-attribute; otherwise, the property
name is ignored). 
> The value of the property must be a valid JsonPath expression. If the JsonPath evaluates
to a JSON array or JSON object and the Return Type is set to 'scalar' the FlowFile will be
unmodified and will be routed to failure. A Return Type of JSON can return scalar values if
the provided JsonPath evaluates to the specified value and will be routed as a match. If Destination
is 'flowfile-content' and the JsonPath does not evaluate to a defined path, the FlowFile will
be routed to 'unmatched' without having its contents modified. If Destination is flowfile-attribute
and the expression matches nothing, attributes will be created with empty strings as the value,
and the FlowFile will always be routed to 'matched.'
> {panel}
> {panel:title=SplitJson}
> Splits a JSON File into multiple, separate FlowFiles for an array element specified by
a JsonPath expression. Each generated FlowFile is comprised of an element of the specified
array and transferred to relationship 'split, with the original file transferred to the 'original'
relationship. If the specified JsonPath is not found or  does not evaluate to an array element,
the original file is routed to 'failure' and no files are generated.
> {panel}
> One item of note is the transitive dependency of ASM through Json-Smart through JsonPath.
> I have included, what I believe is needed to appropriately make use of this item in the
LICENSE.  Review of its correctness would is requested.
> Any feedback is appreciated.  Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message