avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (AVRO-642) Prettify the json dumped during exceptions while validating a writer/reader schema
Date Tue, 26 Oct 2010 20:28:22 GMT

    [ https://issues.apache.org/jira/browse/AVRO-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925120#action_12925120
] 

Doug Cutting commented on AVRO-642:
-----------------------------------

I now see the following error when I run 'cd lang/py; ant clean test'.

{code}
  [py-test] ======================================================================
  [py-test] ERROR: test_type_exception (test_io.TestIO)
  [py-test] ----------------------------------------------------------------------
  [py-test] Traceback (most recent call last):
  [py-test]   File "/home/cutting/src/avro/trunk/lang/py/build/test/test_io.py", line 334,
in test_type_exception
  [py-test]     self.assertRaises(io.AvroTypeException, write_datum, datum_to_write, writers_schema)
  [py-test]   File "/usr/lib/python2.6/unittest.py", line 336, in failUnlessRaises
  [py-test]     callableObj(*args, **kwargs)
  [py-test]   File "/home/cutting/src/avro/trunk/lang/py/build/test/test_io.py", line 119,
in write_datum
  [py-test]     datum_writer.write(datum, encoder)
  [py-test]   File "/home/cutting/src/avro/trunk/lang/py/build/src/avro/io.py", line 755,
in write
  [py-test]     raise AvroTypeException(self.writers_schema, datum)
  [py-test]   File "/home/cutting/src/avro/trunk/lang/py/build/src/avro/io.py", line 83, in
__init__
  [py-test]     pretty_datum = json.dumps(json.loads(str(datum)), indent=2)
  [py-test]   File "/usr/lib/python2.6/json/__init__.py", line 307, in loads
  [py-test]     return _default_decoder.decode(s)
  [py-test]   File "/usr/lib/python2.6/json/decoder.py", line 319, in decode
  [py-test]     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  [py-test]   File "/usr/lib/python2.6/json/decoder.py", line 336, in raw_decode
  [py-test]     obj, end = self._scanner.iterscan(s, **kw).next()
  [py-test]   File "/usr/lib/python2.6/json/scanner.py", line 55, in iterscan
  [py-test]     rval, next_pos = action(m, context)
  [py-test]   File "/usr/lib/python2.6/json/decoder.py", line 171, in JSONObject
  [py-test]     raise ValueError(errmsg("Expecting property name", s, end))
  [py-test] ValueError: Expecting property name: line 1 column 1 (char 1)
 {code}

> Prettify the json dumped during exceptions while validating a writer/reader schema
> ----------------------------------------------------------------------------------
>
>                 Key: AVRO-642
>                 URL: https://issues.apache.org/jira/browse/AVRO-642
>             Project: Avro
>          Issue Type: Improvement
>          Components: java, python
>    Affects Versions: 1.3.3
>         Environment: Avro-Trunk
>            Reporter: Harsh J Chouraria
>            Assignee: Harsh J Chouraria
>            Priority: Minor
>             Fix For: 1.5.0
>
>         Attachments: avro.java.schema.pretty.r1.diff, avro.py.schema.pretty.r1.diff,
avro.schema.pretty.r2.diff, avro.schema.pretty.r3.diff
>
>   Original Estimate: 0.02h
>  Remaining Estimate: 0.02h
>
> Right now the exception that's thrown when a reader and written schema don't match (while
reading a file), looks like this:
> {noformat}
> Found {"type":"record","name":"Test","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]},
expecting {"type":"record","name":"Test1","fields":[{"name":"stringField","type":"string"},{"name":"longField","type":"long"}]}
> {noformat}
> While this is perfectly okay for small schema with a few fields, it gets hard to debug
for complex and large ones.
> The attached patch changes the exception message to look like:
> {noformat}
> Found {
>   "type" : "record",
>   "name" : "Test",
>   "fields" : [ {
>     "name" : "stringField",
>     "type" : "string"
>   }, {
>     "name" : "longField",
>     "type" : "long"
>   } ]
> }, expecting {
>   "type" : "record",
>   "name" : "Test1",
>   "fields" : [ {
>     "name" : "stringField",
>     "type" : "string"
>   }, {
>     "name" : "longField",
>     "type" : "long"
>   } ]
> }
> {noformat}
> P.s. Ideally a diff-like message would help too, but its mileage may vary. I guess with
the pretty output printed, a user may easily do a diff (since they now have newlines), so
us providing this is not _really_ needed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message