avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew Hayes (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-1741) Python 3 code should check if avro.codec is None before decoding
Date Tue, 15 Sep 2015 23:39:45 GMT

    [ https://issues.apache.org/jira/browse/AVRO-1741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14746502#comment-14746502
] 

Matthew Hayes commented on AVRO-1741:
-------------------------------------

Result of "mvn test" below.  It's not clear to me if this exercises the Python code.  But,
I tested it separately and it fixed my problem.

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Avro Toplevel ............................... SUCCESS [  1.683 s]
[INFO] Apache Avro Java ................................... SUCCESS [  3.750 s]
[INFO] Apache Avro ........................................ SUCCESS [ 33.808 s]
[INFO] Apache Avro Compiler ............................... SUCCESS [  4.134 s]
[INFO] Apache Avro Maven Plugin ........................... SUCCESS [  8.210 s]
[INFO] Apache Avro IPC .................................... SUCCESS [ 38.122 s]
[INFO] Trevni Java ........................................ SUCCESS [  0.122 s]
[INFO] Trevni Java Core ................................... SUCCESS [  3.216 s]
[INFO] Apache Avro Mapred API ............................. SUCCESS [01:17 min]
[INFO] Trevni Java Avro ................................... SUCCESS [ 12.650 s]
[INFO] Trevni Specification ............................... SUCCESS [  0.310 s]
[INFO] Apache Avro Tools .................................. SUCCESS [ 18.924 s]
[INFO] Apache Avro Protobuf Compatibility ................. SUCCESS [  0.933 s]
[INFO] Apache Avro Thrift Compatibility ................... SUCCESS [  1.197 s]
[INFO] Apache Avro Maven Archetypes ....................... SUCCESS [  0.668 s]
[INFO] Apache Avro Maven Service Archetype ................ SUCCESS [  0.169 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 03:33 min
[INFO] Finished at: 2015-09-15T16:37:59-07:00
[INFO] Final Memory: 87M/951M
[INFO] ------------------------------------------------------------------------

> Python 3 code should check if avro.codec is None before decoding
> ----------------------------------------------------------------
>
>                 Key: AVRO-1741
>                 URL: https://issues.apache.org/jira/browse/AVRO-1741
>             Project: Avro
>          Issue Type: Bug
>            Reporter: Matthew Hayes
>            Assignee: Matthew Hayes
>             Fix For: 1.7.7
>
>         Attachments: AVRO-1741.diff
>
>
> I'm getting the error below when trying to read an Avro file in Python 3.  FWIW, I generated
this Avro file from Pig.  I don't know if that makes a difference.
> {code}
> avro.datafile.DataFileReader(open("test.avro","rb"),avro.io.DatumReader())
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/Users/mhayes/.pyenv/versions/3.4.2/lib/python3.4/site-packages/avro/datafile.py",
line 352, in __init__
>     self.codec = self.GetMeta('avro.codec').decode('utf-8')
> AttributeError: 'NoneType' object has no attribute 'decode'
> {code}
> It appears the problem is in the following code.  It should check if GetMeta returns
None before trying to decode it.  The Python 2 code doesn't have this problem because it is
already decoded.
> {code}
>     # ensure codec is valid
>     self.codec = self.GetMeta('avro.codec').decode('utf-8')
>     if self.codec is None:
>       self.codec = "null"
>     if self.codec not in VALID_CODECS:
>       raise DataFileException('Unknown codec: %s.' % self.codec)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message