nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark Payne (JIRA)" <j...@apache.org>
Subject [jira] [Created] (NIFI-6369) ValidateRecord routes JSON records to 'invalid' when validating a timestamp, even if format is valid
Date Mon, 10 Jun 2019 14:28:00 GMT
Mark Payne created NIFI-6369:
--------------------------------

             Summary: ValidateRecord routes JSON records to 'invalid' when validating a timestamp,
even if format is valid
                 Key: NIFI-6369
                 URL: https://issues.apache.org/jira/browse/NIFI-6369
             Project: Apache NiFi
          Issue Type: Bug
          Components: Extensions
            Reporter: Mark Payne


If a JSON Reader (either JsonPath or JsonTree reader) is used with ValidateRecord and the
schema indicates that a field is a timestamp (e.g.,

{"name": "timestamp", "type": \{"type": "long", "logicalType":"timestamp-millis"}}

) with the JSON having a value like "2019/06/10 12:00:00Z" and a format of "yyyy/MM/dd HH:mm:ss'Z'"
The processor considers this invalid, even when setting "strict type checking" to "false".

This appears to be due to the way that ValidateRecord calls RecordReader.nextReader() - it
calls it "reader.nextRecord( /* coerce types **/* false, /* drop unknown types **/* false)" 

When this is called, the Record Reader checks the 'coerce types' flag and determines that
it should not coerce the string "2019/06/10 12:00:00Z" into a timestamp. However, for any
date/time/timestamp fields, if the value is a String and it matches the configured pattern,
the Record Readers should go ahead and convert the value into the appropriate java.sql.Date,
java.sql.Time, java.sql.Timestamp object because that's clearly the intent of the schema and
formatting being applied.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message