nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (Jira)" <j...@apache.org>
Subject [jira] [Commented] (NIFI-6369) ValidateRecord routes JSON records to 'invalid' when validating a timestamp, even if format is valid
Date Fri, 20 Sep 2019 03:26:00 GMT

    [ https://issues.apache.org/jira/browse/NIFI-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16934006#comment-16934006
] 

ASF subversion and git services commented on NIFI-6369:
-------------------------------------------------------

Commit e18d9ce1e8d576a21630dcd67e9260ff5c3f9bf0 in nifi's branch refs/heads/master from Mark
Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=e18d9ce ]

NIFI-6369: This closes #3560. Updated JSON Readers to convert String values to Date/Time/Timestamp
objects when appropriate according to the schema and the configured pattern
NIFI-6939: Fix to WriteJsonRecord to deal with date/time/timestamp when no format is explicitly
set


> ValidateRecord routes JSON records to 'invalid' when validating a timestamp, even if
format is valid
> ----------------------------------------------------------------------------------------------------
>
>                 Key: NIFI-6369
>                 URL: https://issues.apache.org/jira/browse/NIFI-6369
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>            Reporter: Mark Payne
>            Assignee: Mark Payne
>            Priority: Major
>             Fix For: 1.10.0
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> If a JSON Reader (either JsonPath or JsonTree reader) is used with ValidateRecord and
the schema indicates that a field is a timestamp (e.g.,
> {"name": "timestamp", "type": \{"type": "long", "logicalType":"timestamp-millis"}}
> ) with the JSON having a value like "2019/06/10 12:00:00Z" and a format of "yyyy/MM/dd
HH:mm:ss'Z'" The processor considers this invalid, even when setting "strict type checking"
to "false".
> This appears to be due to the way that ValidateRecord calls RecordReader.nextReader()
- it calls it "reader.nextRecord( /* coerce types **/* false, /* drop unknown types **/* false)" 
> When this is called, the Record Reader checks the 'coerce types' flag and determines
that it should not coerce the string "2019/06/10 12:00:00Z" into a timestamp. However, for
any date/time/timestamp fields, if the value is a String and it matches the configured pattern,
the Record Readers should go ahead and convert the value into the appropriate java.sql.Date,
java.sql.Time, java.sql.Timestamp object because that's clearly the intent of the schema and
formatting being applied.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message