avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miroslav Csonka (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AVRO-1886) Improve error reporting for schema validation
Date Tue, 26 Jul 2016 09:39:20 GMT

     [ https://issues.apache.org/jira/browse/AVRO-1886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Miroslav Csonka updated AVRO-1886:
----------------------------------
    Description: 
In Funding Circle we use avro schema to validate some financial data and we've found ourselves
wondering what is invalid. We've noticed that avro [does already validation|https://github.com/apache/avro/blob/master/lang/ruby/lib/avro/schema.rb#L94],
but does not report what fields are invalid, so we've decided to change it and we use Avro
as a validation library.

h5. What error messages to expect?

We communicate errors for a given payload in format "at <path> <expected> <received>".
For <path> we've decided to use syntax similar to json path:
* . refers to root
* \[0\] first index in array
* .age would match 42 from {"age": 42}


Error within object *person* and his optional array *houses* on *second item* with key *number_of_rooms*:
{quote}
    at .person.houses\[1\].number_of_rooms expected type long, got string with value "not
valid at all"
{quote}

Example of multiple errors for a single payload:
{quote}
    at .\[0\] expected type int, got null
    at .\[1\] expected type int, got string with value \"e\"
{quote}

More examples can be seen at https://github.com/apache/avro/pull/111/files#diff-830ac80276854fceefa3f92c346d165f

  was:
In Funding Circle we use avro schema to validate some financial data and we've found ourselves
wondering what is invalid. We've noticed that avro [does already validation|https://github.com/apache/avro/blob/master/lang/ruby/lib/avro/schema.rb#L94],
but does not report what fields are invalid.

 At this point, we can clearly communicate validation scenarios and avro can be even used
as a validation library.

Little preview of what we've done
{quote}
    at .person.houses\[1\].number_of_rooms expected type long, got string with value "not
valid at all"
{quote}

At this point, our fork is quite outdated so it will take us some time to update it and submit
a patch. This is just a question if you would be interested in receiving such a patch.


> Improve error reporting for schema validation
> ---------------------------------------------
>
>                 Key: AVRO-1886
>                 URL: https://issues.apache.org/jira/browse/AVRO-1886
>             Project: Avro
>          Issue Type: Improvement
>          Components: ruby
>            Reporter: Miroslav Csonka
>            Assignee: Miroslav Csonka
>             Fix For: 1.9.0, 1.8.2
>
>
> In Funding Circle we use avro schema to validate some financial data and we've found
ourselves wondering what is invalid. We've noticed that avro [does already validation|https://github.com/apache/avro/blob/master/lang/ruby/lib/avro/schema.rb#L94],
but does not report what fields are invalid, so we've decided to change it and we use Avro
as a validation library.
> h5. What error messages to expect?
> We communicate errors for a given payload in format "at <path> <expected>
<received>".
> For <path> we've decided to use syntax similar to json path:
> * . refers to root
> * \[0\] first index in array
> * .age would match 42 from {"age": 42}
> Error within object *person* and his optional array *houses* on *second item* with key
*number_of_rooms*:
> {quote}
>     at .person.houses\[1\].number_of_rooms expected type long, got string with value
"not valid at all"
> {quote}
> Example of multiple errors for a single payload:
> {quote}
>     at .\[0\] expected type int, got null
>     at .\[1\] expected type int, got string with value \"e\"
> {quote}
> More examples can be seen at https://github.com/apache/avro/pull/111/files#diff-830ac80276854fceefa3f92c346d165f



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message