drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Altekruse (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4056) Avro deserialization corrupts data
Date Sat, 14 Nov 2015 00:29:11 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15004963#comment-15004963
] 

Jason Altekruse commented on DRILL-4056:
----------------------------------------

As I mentioned on the thread about this issue, I believe I have found a fix. There are a lot
of new things in master that are the start of the 1.4 release. For now I am working on top
of 1.3. Please build this branch to try out the fix. I will be adding unit tests before opening
a PR.

https://github.com/jaltekruse/incubator-drill/commits/4056-avro-corruption-bug

> Avro deserialization corrupts data
> ----------------------------------
>
>                 Key: DRILL-4056
>                 URL: https://issues.apache.org/jira/browse/DRILL-4056
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - Other
>    Affects Versions: 1.3.0
>         Environment: Ubuntu 15.04 - Oracle Java
>            Reporter: Stefán Baxter
>             Fix For: 1.3.0
>
>         Attachments: test.zip
>
>
> I have an Avro file that support the following data/schema:
> {"field":"some", "classification":{"variant":"Gæst"}}
> When I select 10 rows from this file I get:
> +---------------------+
> |       EXPR$0        |
> +---------------------+
> | Gæst                |
> | Voksen              |
> | Voksen              |
> | Invitation KIF KBH  |
> | Invitation KIF KBH  |
> | Ordinarie pris KBH  |
> | Ordinarie pris KBH  |
> | Biljetter 200 krBH  |
> | Biljetter 200 krBH  |
> | Biljetter 200 krBH  |
> +---------------------+
> The bug is that the field values are incorrectly de-serialized and the value from the
previous row is retained if the subsequent row is shorter.
> The sql query:
> "select s.classification.variant variant from dfs.<some> as s limit 10;"
> That way the  "Ordinarie pris" becomes "Ordinarie pris KBH" because the previous row
had the value "Invitation KIF KBH".



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message