nifi-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [nifi] simonbence opened a new pull request #4186: NIFI-7300 Allowing narrow numeric types to fit againt schema check with wider type
Date Mon, 06 Apr 2020 12:56:08 GMT
simonbence opened a new pull request #4186: NIFI-7300 Allowing narrow numeric types to fit
againt schema check with wider type
URL: https://github.com/apache/nifi/pull/4186
 
 
   [https://issues.apache.org/jira/browse/NIFI-7300](https://issues.apache.org/jira/browse/NIFI-7300)
   
   As the JSON files are read without considering the schema (used for validation), it might
happen that a field which we expect to be a long is parsed into an integer based on its current
value. During validation the AVRO record incorrectly ends up as invalid (because of an integer
is not considered as long in this example). This is a fix for allow narrow numeric data types
for fit into wider types during schema based validation.
   
   Example:
   
   Existing behaviour:
   - Step 1.: incoming value is "5"
   - Step 2.: record is being read, the representing object will be an Integer
   - Step 3.: the record is validated against a schema which expect the field to be a Long
   - Step 4.: the record is considered as invalid as the Integer field is not considered as
valid Long
   
   Proposed behaviour:
   - Step 1-3.: the same as the existing behaviour
   - Step 4.: as Integer is considered as a narrow data type of Long, the record is considered
as valid (if there are no other discrepancies in the record)
   
   Implementation:
   Building on top of `RecordFieldType's` existing "narrowDataTypes" collection, in case of
numeric types, I extended the validation check with the check of these narrow types: if the
incoming value fits to the original, or any of the narrow types, it is considered as valid.
   
   Note:
   Even if String also has a numerous amount of narrow types for example, in order to keep
compatibility I did not extended other data types with this fix, only numeric ones. Extending
them would be a change of behaviour instead of a fix. See `TestValidateRecord#testValidateMissingRequiredArray`
which for example breaks if narrow types would be allowed for Strings.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
        in the commit message?
   
   - [ ] Does your PR title start with **NIFI-XXXX** where XXXX is the JIRA number you are
trying to resolve? Pay particular attention to the hyphen "-" character.
   
   - [ ] Has your PR been rebased against the latest commit within the target branch (typically
`master`)?
   
   - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response
to PR reviewer feedback should be made on this branch and pushed to allow change tracking.
Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check
clean install` at the root `nifi` folder?
   - [ ] Have you written or updated unit tests to verify your changes?
   - [ ] Have you verified that the full build is successful on both JDK 8 and JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies licensed in a way
that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?

   - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE`
file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file
found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic
access) for each of the new properties?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check travis-ci for build issues and submit
an update to your PR as soon as possible.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message