avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AVRO-820) Java: Exceptions thrown while encoding a record while writing an Avro Data file will produce a corrupt file.
Date Mon, 16 May 2011 13:46:47 GMT

    [ https://issues.apache.org/jira/browse/AVRO-820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034016#comment-13034016
] 

Doug Cutting commented on AVRO-820:
-----------------------------------

I consider it an error to continue using a file after writing to it had thrown an exception.
 So an alternate fix might be to close the file or otherwise prohibit further action on it
after an exception is thrown, although even that seems redundant to me.  How is an application
to know whether an exception was thrown by an encoder and not by the filesystem or somesuch?
 Suggesting that folks should continue writing after an exception has been thrown seems dangerous
to me.

> Java: Exceptions thrown while encoding a record while writing an Avro Data file will
produce a corrupt file. 
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: AVRO-820
>                 URL: https://issues.apache.org/jira/browse/AVRO-820
>             Project: Avro
>          Issue Type: Bug
>          Components: java
>    Affects Versions: 1.4.1, 1.5.0, 1.5.1
>            Reporter: Scott Carey
>            Assignee: Scott Carey
>            Priority: Critical
>         Attachments: AVRO-820.patch
>
>
> If an exception is thrown while serializing a record in DataFileWriter<D>.append(D)
partial contents of that serialization will end up in the file.  This corrupts the block.
 
> DataFileWriter should ensure that the buffer is rewound to the state prior to the record
write in the case of an exception thrown during serialization to prevent creating a corrupt
file.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message